-- Logs begin at Wed 2022-07-27 15:49:16 UTC, end at Wed 2022-07-27 16:55:41 UTC. -- Jul 27 16:21:31.848084 dv-cinder-849690-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 27 16:21:31.858154 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 27 16:21:31.860793 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 27 16:21:31 2022] *** Jul 27 16:21:31.860793 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 27 16:21:31.860793 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 27 16:21:31.860793 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: nodename: dv-cinder-849690-6-iscsi Jul 27 16:21:31.860793 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: machine: x86_64 Jul 27 16:21:31.860793 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: clock source: unix Jul 27 16:21:31.860793 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: pcre jit disabled Jul 27 16:21:31.860793 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: detected number of CPU cores: 4 Jul 27 16:21:31.860793 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: current working directory: / Jul 27 16:21:31.860793 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: detected binary path: /usr/bin/uwsgi-core Jul 27 16:21:31.861446 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: your processes number limit is 47680 Jul 27 16:21:31.861446 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: your memory page size is 4096 bytes Jul 27 16:21:31.861446 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: detected max file descriptor number: 2048 Jul 27 16:21:31.861446 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: lock engine: pthread robust mutexes Jul 27 16:21:31.861446 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: thunder lock: enabled Jul 27 16:21:31.861446 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 27 16:21:31.861446 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 27 16:21:31.900088 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: Python main interpreter initialized at 0x55ecde1e89c0 Jul 27 16:21:31.900088 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: python threads support enabled Jul 27 16:21:31.900088 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: your server socket listen backlog is limited to 100 connections Jul 27 16:21:31.900088 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: your mercy for graceful operations on workers is 90 seconds Jul 27 16:21:31.900418 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: mapped 403077 bytes (393 KB) for 2 cores Jul 27 16:21:31.900494 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: *** Operational MODE: preforking *** Jul 27 16:21:31.900755 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: *** uWSGI is running in multiple interpreter mode *** Jul 27 16:21:31.900755 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: spawned uWSGI master process (pid: 80105) Jul 27 16:21:31.900948 dv-cinder-849690-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 27 16:21:31.901944 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: spawned uWSGI worker 1 (pid: 80109, cores: 1) Jul 27 16:21:31.901944 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: spawned uWSGI worker 2 (pid: 80110, cores: 1) Jul 27 16:21:31.901944 dv-cinder-849690-6-iscsi devstack@keystone.service[80105]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 27 16:21:37.384532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 16:21:37.384532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: from cryptography.utils import int_from_bytes Jul 27 16:21:37.385353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 16:21:37.385353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: from cryptography.utils import int_from_bytes Jul 27 16:21:37.385743 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 16:21:37.385743 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: from cryptography.utils import int_from_bytes Jul 27 16:21:37.386194 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 16:21:37.386194 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: from cryptography.utils import int_from_bytes Jul 27 16:21:37.434871 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 27 16:21:37.435030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 27 16:21:37.435115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 27 16:21:37.435200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 27 16:21:37.435278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] command line args: [] {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 27 16:21:37.435350 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] command line args: [] {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 27 16:21:37.435458 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 27 16:21:37.435593 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 27 16:21:37.435678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ================================================================================ {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 27 16:21:37.435776 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ================================================================================ {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 27 16:21:37.435887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] admin_token = **** {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.436031 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] admin_token = **** {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.436120 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] config_dir = [] {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.436278 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] config_dir = [] {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.436361 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.436489 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.436575 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] config_source = [] {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.437250 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] debug = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.437337 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] config_source = [] {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.437454 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.437753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] debug = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.437753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.437978 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.438082 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.438183 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.438279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.438379 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.438481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.438577 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.438677 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.438784 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] list_limit = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.438878 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] list_limit = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.438976 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] log_config_append = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.439082 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] log_config_append = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.439183 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.439291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.439386 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] log_dir = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.439484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] log_dir = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.439579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] log_file = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.439690 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] log_file = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.439770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.439893 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.439974 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.440052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.440167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.440257 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.440337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.440410 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.440536 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.440613 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.440737 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.440821 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.440925 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.441000 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.441129 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.441206 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.441334 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.441416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.441544 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.441632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.441779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.441900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.442014 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.442094 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.442259 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.442345 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.442503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.442620 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.442773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.442851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.443270 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.74/identity {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.443347 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.74/identity {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.443474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] publish_errors = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.443555 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] publish_errors = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.443673 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.443756 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.443901 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.443977 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.444099 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.444173 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.444315 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.444469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.444556 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.444618 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.444770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] standard_threads = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.444865 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] standard_threads = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.445030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.445095 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.445253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.445338 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.445416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.445486 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.445588 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] use_journal = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.445673 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] use_journal = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.445750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] use_json = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.445825 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] use_json = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.445921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] use_stderr = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.445995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] use_stderr = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.446097 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] use_syslog = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.446239 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] use_syslog = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.446318 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.446394 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 16:21:37.446565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.446666 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.446801 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.446879 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.447056 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.447132 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.447292 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.447376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.447540 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.447623 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.447777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.447854 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.448023 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.448099 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.448257 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.448335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.448502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.448587 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.448746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.448823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.449018 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.449097 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.449255 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.449332 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.449500 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.449577 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.449736 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.449854 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.449970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.450047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.450481 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.450566 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.450716 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.450790 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.450957 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.451042 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.451207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.451284 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.451449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.451527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.451684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.451760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.451934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.452012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.452218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.452299 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.452470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.452546 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.452716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.452792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.452969 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.453056 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.453202 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.453289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.453431 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.453517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.453665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.453742 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.453911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.453987 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.454155 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.454247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.454384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.454461 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.454612 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.454687 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.454854 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.454938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.455273 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.455346 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.455416 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.455486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.455555 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.455677 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.455763 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.455833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.455996 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.456081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.456229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.456379 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.456454 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.456523 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.456674 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.456752 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.456911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.457002 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.457144 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.457221 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.457378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.457455 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.457624 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.457709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.457853 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.457935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.458092 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.458176 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.458320 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.458408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.458556 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.458638 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.458788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.458893 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.459033 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.459144 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.459263 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.459365 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.459497 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.459600 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.459731 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.459868 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.459987 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.460062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.460177 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.460259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.460402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.460492 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.460631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.460714 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.460876 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.460998 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.461133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.461212 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.461368 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.461450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.461598 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.461689 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.461852 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.461931 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.462088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.462198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.462322 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.462431 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.462559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.462700 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.462786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.462904 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.463041 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.463128 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.463276 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.463396 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.463522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.463643 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.463767 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.463844 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.463999 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.464076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.464233 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.464314 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.464466 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.464547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.464710 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.464788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.464968 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.465051 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.465205 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.465282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.465447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.465524 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.465682 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.465761 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.465917 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.465994 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.466151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.466228 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.466384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.466463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.466617 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.466696 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.466875 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.466953 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.467114 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.467198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.467352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.467429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.467589 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.467670 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.467855 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.467936 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.468092 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.468171 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.468329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.468406 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.468608 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.468698 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.468842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.468966 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.469103 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.469182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.469334 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.469413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.469569 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.469649 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.469800 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.469881 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.470039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.470125 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.470272 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.470350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.470520 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.470598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.470757 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.470844 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.471010 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.471089 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.471268 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.471347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.471508 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.471594 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.471740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] auth.external = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.471818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] auth.external = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.471970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.472054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.472266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.472376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.472496 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.472584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.472727 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] auth.password = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.472806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] auth.password = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.472979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] auth.token = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.473064 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] auth.token = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.473222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.473315 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.473459 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.473539 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.473707 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.473786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.473940 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.474027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.474188 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.474267 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.474435 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.474513 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.474664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.474750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.474899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] credential.caching = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.474978 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] credential.caching = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.475143 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.475221 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.475388 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.475475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.475636 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.475715 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.475870 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.475949 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.476116 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.476207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.476350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.476429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.476619 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.476698 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.476871 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.477027 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.477137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.477219 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.477388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.477466 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.477668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.477755 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.477910 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.477988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.478176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.478255 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.478417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.478504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.478673 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.478752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.478906 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] federation.caching = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.478984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] federation.caching = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.479145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.479237 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.479393 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.479473 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.479638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.479715 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.479868 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.479955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.480115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.480199 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.480351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.480430 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.480603 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.480693 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.480839 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.480963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.481116 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.481195 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.481355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.481435 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.481603 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.481681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.481838 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] identity.caching = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.481916 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] identity.caching = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.482085 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.482165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.482330 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.482408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.482564 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.482642 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.482802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.482883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.483047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.483125 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.483278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.483857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.484040 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.484121 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.484289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.484368 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.484539 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.484617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.484771 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.484850 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.485025 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.485105 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.485255 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.485333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.485498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.485578 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.485742 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.485820 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.485987 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.486065 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.486239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.486324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.486484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.486562 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.486751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.486828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.486987 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.487066 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.487221 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.487305 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.487451 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.487530 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.487686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.487765 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.487917 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.487996 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.488150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.488228 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.488389 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.488467 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.488630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.488710 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.488873 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.488984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.489127 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.489208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.489372 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.489450 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.489615 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.489693 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.489848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.489926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.490092 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.490170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.490337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.490415 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.490566 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.490646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.490802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.490880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.491029 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.491107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.491265 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.491344 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.491500 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.491578 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.491741 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.491819 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.491977 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.492056 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.492214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.492292 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.492461 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.492539 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.492707 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.492787 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.492959 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.493041 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.493192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.493276 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.493440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.493520 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.493686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.493764 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.493918 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.494005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.494152 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.494230 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.494386 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.494463 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.494616 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.user = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.494702 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.user = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.494854 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.494932 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.495109 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.495194 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.495341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.495427 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.495584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.495661 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.495828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.495906 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.496072 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.496157 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.496305 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.496384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.496534 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.496611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.496766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.496852 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.497024 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.497104 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.497260 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.497339 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.497486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.497572 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.497729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.497807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.497971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.498050 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.498215 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.498301 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.498465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.498574 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.498709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.498827 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.498958 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.499638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.499822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.499903 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.500066 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.500146 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.500306 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.500394 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.500544 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.500624 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.500798 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.500906 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.501071 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.501164 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.501322 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.501402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.501561 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.501641 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.501816 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.501903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.502048 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.502127 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.502293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.502373 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.502531 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.502619 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.502767 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.502846 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.503004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.503083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.503250 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.503336 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.503491 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.503569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.503721 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.503799 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.503959 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.504047 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.504198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] resource.caching = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.504276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] resource.caching = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.504445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.504523 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.504691 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.504777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.504947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.505030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.505200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.505280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.505452 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.505544 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.505690 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.505769 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.505937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.506023 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.506176 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.506326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.506402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.506473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.506636 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] role.caching = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.506716 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] role.caching = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.506879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] role.driver = sql {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.506966 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] role.driver = sql {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.507107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.507185 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.507354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.507433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.507598 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.507686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.507842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.507923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.508087 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.508165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.508331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.508411 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.508574 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.508652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.508817 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.508944 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.509086 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.509166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.509317 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.509395 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.509563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.509643 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.509811 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.509890 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.510056 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.510133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.510300 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.510378 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.510708 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.510788 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.510944 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.511022 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.511196 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.511275 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.511445 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.511526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.511693 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.511771 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.511941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.512019 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.512175 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.512255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.512428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.512506 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.512680 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.512758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.512937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.513019 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.513173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.513252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.513403 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.513493 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.513632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.513712 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.513881 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.513959 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.514135 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.514221 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.514383 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.514461 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.514646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.514726 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.514885 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.514972 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.515116 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.515195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.515351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] token.caching = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.515429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] token.caching = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.515587 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.515677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.515833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.515911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.516068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.516146 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.516321 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.516407 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.516567 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.516646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.516804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.516882 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.517072 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.517164 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.517315 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.517393 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.517559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.517637 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.517795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.517880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.518030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.518108 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.518264 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.518341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.518507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.518592 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.518750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.518828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.518977 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.519055 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.519216 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.519302 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.519503 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.519583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.519765 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] database.connection = **** {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.519853 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] database.connection = **** {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.520007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.520097 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.520251 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.520329 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.520488 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.520567 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.520725 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.520804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.520981 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.521063 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.521218 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.521297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.521452 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.521530 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.521685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.521763 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.521928 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.522007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.522161 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.522239 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.522403 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.522481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.522636 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.522716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.522883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.522962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.523120 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.523198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.523362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.523441 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.523591 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.523669 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.523827 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.523905 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.524061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.524140 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 16:21:37.524213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80109) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 27 16:21:37.524283 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 27 16:21:37.553804 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.553942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.554037 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.554115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.554249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.554336 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.563073 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.563332 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.593754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.593921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.593991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.594110 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.651181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.651487 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.703434 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.703667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.706478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.706718 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.769081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.769310 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.769395 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.769535 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.769638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.769758 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.769843 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.769975 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.770058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.770186 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.770279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.770398 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.793654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.793761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.793884 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.793984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.794103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.794186 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.794317 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.794400 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.794534 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.794618 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.794743 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.794825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.794953 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.795044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.806449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.806735 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.819608 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.819779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.819866 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.820014 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.820097 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.820237 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.820320 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.820447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.820538 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.820658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.820741 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.820868 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.820969 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80109) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.821105 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 16:21:37.838226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.838435 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 16:21:37.838772 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.838971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 16:21:37.839053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.839236 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 16:21:37.839317 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.839491 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.839571 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.839747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.839837 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 16:21:37.840002 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.840083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.840258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.840515 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.840769 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.850194 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.850470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.850739 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.851005 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.851086 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.851281 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.851549 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.851825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.857900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80109) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.858786 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.862615 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80109) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.862895 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.863177 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.863433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.863515 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.863713 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.863808 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.863974 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.864054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.864240 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.864321 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.864510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.864593 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.864776 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.864854 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.865073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.865154 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.865339 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.865421 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.865610 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.865883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.866152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.888635 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.889562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.890210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80109) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.890489 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.891074 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.891352 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.895101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80109) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.895381 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.895665 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.895942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.896221 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.896504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.902252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80109) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.902531 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.903129 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.903406 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.906668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80109) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.906964 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.907249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.907530 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.907646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.907803 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.907933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.908086 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.908230 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.908346 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.908505 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.908616 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.908774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.909081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.909360 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.909640 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.925067 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.925342 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.926102 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.926387 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.929285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80109) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.930326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.932393 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.932696 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.933502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.933795 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.937172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80109) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.938260 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.940336 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80109) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.941464 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.943543 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.943843 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.944139 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.944432 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.944593 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.944714 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.944914 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.945026 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.945196 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.945498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.945784 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.946071 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.961824 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80109) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.962056 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.962324 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.962589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.962902 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.963186 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.963451 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.963711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.970559 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.971239 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.972234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.972455 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.972808 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.973107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.975911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.976003 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80109) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.976189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.976273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.976494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.976577 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.976776 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.976868 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.977088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.977173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.988728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.988826 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80109) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:37.989030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.989116 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.989309 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.989392 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.989588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.989671 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.989867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.989949 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.990144 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.990227 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.990417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:37.990498 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.005999 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80109) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:38.006093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:38.008971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80109) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:38.009078 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:38.011919 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.012003 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.014498 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.015085 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.016531 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80109) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:38.016678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:38.016814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.017000 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.017123 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.017283 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.023621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.023714 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.025234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80109) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:38.025345 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:38.028161 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.028263 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.028451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.028534 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.028732 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.028815 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.029037 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.029121 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.037670 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80109) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:38.037763 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:38.037954 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.038036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.038230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.038312 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.045078 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:38.045165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80109) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 16:21:38.045359 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.045455 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.045639 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.045722 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.045933 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.046014 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.046218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.046300 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.046486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.046567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.046756 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.046837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.047029 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.047121 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.047406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.047496 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.047686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.047768 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.047962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.048043 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.048223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.048310 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.048486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.048567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.048742 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80109) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.048824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 16:21:38.082907 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55ecde1e89c0 pid: 80110 (default app) Jul 27 16:21:38.083008 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55ecde1e89c0 pid: 80109 (default app) Jul 27 16:21:38.086185 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86268c-2dff-4c8a-a4a6-7d1d7e3eb592 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:38.086479 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86268c-2dff-4c8a-a4a6-7d1d7e3eb592 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:38.086757 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86268c-2dff-4c8a-a4a6-7d1d7e3eb592 None None] PATH_INFO: `/v3` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:38.091061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1/1] 10.222.0.74 () {50 vars in 791 bytes} [Wed Jul 27 16:21:38 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:21:45.613962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafc783-1d72-4d59-8ead-e588901a299e None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:45.614518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafc783-1d72-4d59-8ead-e588901a299e None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:45.614518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafc783-1d72-4d59-8ead-e588901a299e None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:45.618954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1/2] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:21:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:21:45.624493 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec0f99-ac14-4290-abe4-75854a09ccce None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:45.624924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec0f99-ac14-4290-abe4-75854a09ccce None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:45.625067 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec0f99-ac14-4290-abe4-75854a09ccce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:45.694611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: INFO dbcounter [None req-deec0f99-ac14-4290-abe4-75854a09ccce None None] Registered counter for database keystone Jul 27 16:21:45.718852 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG oslo_db.sqlalchemy.engines [None req-deec0f99-ac14-4290-abe4-75854a09ccce None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80110) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 27 16:21:45.725588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writer thread running {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 27 16:21:45.908015 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG passlib.handlers.bcrypt [None req-deec0f99-ac14-4290-abe4-75854a09ccce None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80110) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 27 16:21:45.908369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG passlib.handlers.bcrypt [None req-deec0f99-ac14-4290-abe4-75854a09ccce None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80110) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 27 16:21:45.963317 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-deec0f99-ac14-4290-abe4-75854a09ccce None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:21:45.964949 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-deec0f99-ac14-4290-abe4-75854a09ccce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:21:46.094114 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 2/3] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:21:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 27 16:21:46.215601 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafc783-1d72-4d59-8ead-e588901a299e None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:46.285439 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: INFO dbcounter [None req-1dafc783-1d72-4d59-8ead-e588901a299e None None] Registered counter for database keystone Jul 27 16:21:46.304682 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG oslo_db.sqlalchemy.engines [None req-1dafc783-1d72-4d59-8ead-e588901a299e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80109) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 27 16:21:46.310903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writer thread running {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 27 16:21:46.428361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafc783-1d72-4d59-8ead-e588901a299e None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:46.432450 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea820c2-b2a9-4dbb-8330-85bbabaaafa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:46.433124 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea820c2-b2a9-4dbb-8330-85bbabaaafa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:46.433409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea820c2-b2a9-4dbb-8330-85bbabaaafa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:46.433685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea820c2-b2a9-4dbb-8330-85bbabaaafa6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:46.441754 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea820c2-b2a9-4dbb-8330-85bbabaaafa6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:46.493519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7ea820c2-b2a9-4dbb-8330-85bbabaaafa6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 27 16:21:46.493860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea820c2-b2a9-4dbb-8330-85bbabaaafa6 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:46.503368 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 2/4] 10.222.0.74 () {58 vars in 1324 bytes} [Wed Jul 27 16:21:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 16:21:46.509448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec0f99-ac14-4290-abe4-75854a09ccce None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:46.523682 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec0f99-ac14-4290-abe4-75854a09ccce None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:46.528104 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d6ff5-4bcd-425e-a893-3441bfbfdb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:46.529248 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d6ff5-4bcd-425e-a893-3441bfbfdb2e admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:46.529532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d6ff5-4bcd-425e-a893-3441bfbfdb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:46.530316 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d6ff5-4bcd-425e-a893-3441bfbfdb2e admin admin] PATH_INFO: `/v3/projects/0d94f0c6f55c441591df5c2cef40a859` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:46.531806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d6ff5-4bcd-425e-a893-3441bfbfdb2e admin admin] RBAC: Authorizing `identity:get_project(project_id=0d94f0c6f55c441591df5c2cef40a859)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:46.582175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f48d6ff5-4bcd-425e-a893-3441bfbfdb2e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 27 16:21:46.582519 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d6ff5-4bcd-425e-a893-3441bfbfdb2e admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:46.587129 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 3/5] 10.222.0.74 () {56 vars in 1207 bytes} [Wed Jul 27 16:21:46 2022] GET /identity/v3/projects/0d94f0c6f55c441591df5c2cef40a859 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:21:48.534867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf17258-fbc5-4059-a8d3-399c17808b25 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:48.535418 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf17258-fbc5-4059-a8d3-399c17808b25 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:48.535418 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf17258-fbc5-4059-a8d3-399c17808b25 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:48.539580 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 3/6] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:21:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:21:48.544961 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017f0f7-0e28-46d0-8b03-3a9d1ad1876d None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:48.545355 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017f0f7-0e28-46d0-8b03-3a9d1ad1876d None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:48.545550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017f0f7-0e28-46d0-8b03-3a9d1ad1876d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:48.565703 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-0017f0f7-0e28-46d0-8b03-3a9d1ad1876d None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:21:48.567213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0017f0f7-0e28-46d0-8b03-3a9d1ad1876d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:21:48.575449 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 4/7] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:21:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 27 16:21:48.598643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf17258-fbc5-4059-a8d3-399c17808b25 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:48.610699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf17258-fbc5-4059-a8d3-399c17808b25 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:48.614917 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdb975-2242-49a4-baa9-272f7ac13a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:48.615526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdb975-2242-49a4-baa9-272f7ac13a95 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:48.615807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdb975-2242-49a4-baa9-272f7ac13a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:48.616083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdb975-2242-49a4-baa9-272f7ac13a95 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:48.624616 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cdb975-2242-49a4-baa9-272f7ac13a95 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:48.626440 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cdb975-2242-49a4-baa9-272f7ac13a95 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:48.635236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 4/8] 10.222.0.74 () {58 vars in 1324 bytes} [Wed Jul 27 16:21:48 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 16:21:48.642260 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017f0f7-0e28-46d0-8b03-3a9d1ad1876d None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:48.652656 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017f0f7-0e28-46d0-8b03-3a9d1ad1876d None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:48.657763 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1a8d9-61d1-409d-8d14-e90e8859c0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:48.658403 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1a8d9-61d1-409d-8d14-e90e8859c0b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:48.658689 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1a8d9-61d1-409d-8d14-e90e8859c0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:48.658972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1a8d9-61d1-409d-8d14-e90e8859c0b7 admin admin] PATH_INFO: `/v3/users/2682f5470265422cbaaa91dfd16e8ea6` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:48.660357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1a8d9-61d1-409d-8d14-e90e8859c0b7 admin admin] RBAC: Authorizing `identity:get_user(user_id=2682f5470265422cbaaa91dfd16e8ea6)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:48.662238 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1a8d9-61d1-409d-8d14-e90e8859c0b7 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:48.663807 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 5/9] 10.222.0.74 () {56 vars in 1198 bytes} [Wed Jul 27 16:21:48 2022] GET /identity/v3/users/2682f5470265422cbaaa91dfd16e8ea6 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:21:50.704327 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196c47f-e768-43cf-a5fe-0ca91e549e3e None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:50.704851 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196c47f-e768-43cf-a5fe-0ca91e549e3e None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:50.704967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196c47f-e768-43cf-a5fe-0ca91e549e3e None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:50.706244 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 5/10] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:21:50.711261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f0b42-b439-4e92-95c6-5f9d7025d90a None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:50.711658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f0b42-b439-4e92-95c6-5f9d7025d90a None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:50.711815 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f0b42-b439-4e92-95c6-5f9d7025d90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:50.731096 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-da9f0b42-b439-4e92-95c6-5f9d7025d90a None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:21:50.733063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-da9f0b42-b439-4e92-95c6-5f9d7025d90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:21:50.733185 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c4e37-cb33-400e-8f76-aa72ae669e62 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:50.733466 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c4e37-cb33-400e-8f76-aa72ae669e62 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:50.733742 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c4e37-cb33-400e-8f76-aa72ae669e62 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:50.735051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 6/11] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:21:50.739800 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfec93-06ed-4b17-b69f-8a95f8ac8460 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:50.740082 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfec93-06ed-4b17-b69f-8a95f8ac8460 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:50.740356 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfec93-06ed-4b17-b69f-8a95f8ac8460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:50.781822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG passlib.handlers.bcrypt [None req-c6dfec93-06ed-4b17-b69f-8a95f8ac8460 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80109) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 27 16:21:50.782252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG passlib.handlers.bcrypt [None req-c6dfec93-06ed-4b17-b69f-8a95f8ac8460 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80109) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 27 16:21:50.813631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 6/12] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:21:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:21:50.828165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-c6dfec93-06ed-4b17-b69f-8a95f8ac8460 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:21:50.829750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-c6dfec93-06ed-4b17-b69f-8a95f8ac8460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:21:50.836307 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f0b42-b439-4e92-95c6-5f9d7025d90a None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:50.859342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:50.929111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 7/13] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:21:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:21:50.938397 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f0b42-b439-4e92-95c6-5f9d7025d90a None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:50.950681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfec93-06ed-4b17-b69f-8a95f8ac8460 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:50.972066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368c87e-db17-4a9a-b4ec-015ad06cbbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:50.972651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368c87e-db17-4a9a-b4ec-015ad06cbbaa None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:50.972951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368c87e-db17-4a9a-b4ec-015ad06cbbaa None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:50.973227 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368c87e-db17-4a9a-b4ec-015ad06cbbaa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:50.984622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368c87e-db17-4a9a-b4ec-015ad06cbbaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:21:50.993326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-c368c87e-db17-4a9a-b4ec-015ad06cbbaa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:21:50.993326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:21:50.993326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:21:50.993326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:21:50.993326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:21:50.993326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:21:50.993326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:21:50.993326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:21:50.993326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:21:50.993326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:21:50.993326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:21:50.993326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:21:50.993326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:21:50.993326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:21:50.993326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:21:50.993326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:21:50.994284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:21:50.994284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:21:50.994284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:21:50.994284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:21:50.994284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:21:50.994284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:21:50.994284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:21:50.994284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:21:50.994284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:21:50.994284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:21:50.994284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:21:50.994284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:21:50.994284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:21:50.994284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:21:50.994284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:21:50.994284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:21:50.995098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:21:50.995098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:21:50.995098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:21:50.995098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:21:50.995098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:21:50.995098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:21:50.995098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:21:50.995098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:21:50.995098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:21:50.995098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:21:50.995098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:21:50.995098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:21:50.995098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 7/14] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:21:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:21:51.002471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368c87e-db17-4a9a-b4ec-015ad06cbbaa None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:51.050912 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfec93-06ed-4b17-b69f-8a95f8ac8460 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:51.053206 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:51.083114 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:51.088651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a124b86-a24c-4178-9b37-778b4ff52f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:51.089556 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a124b86-a24c-4178-9b37-778b4ff52f6e None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:51.089840 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a124b86-a24c-4178-9b37-778b4ff52f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:51.090113 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a124b86-a24c-4178-9b37-778b4ff52f6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:51.103518 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368c87e-db17-4a9a-b4ec-015ad06cbbaa None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:51.141532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97cebd-b8ed-4265-a151-07f69f99dfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:51.142162 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97cebd-b8ed-4265-a151-07f69f99dfb5 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:51.142448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97cebd-b8ed-4265-a151-07f69f99dfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:51.142731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97cebd-b8ed-4265-a151-07f69f99dfb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:51.143461 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97cebd-b8ed-4265-a151-07f69f99dfb5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:21:51.143770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97cebd-b8ed-4265-a151-07f69f99dfb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:51.145199 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97cebd-b8ed-4265-a151-07f69f99dfb5 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:51.158594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 8/15] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:21:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:21:51.159016 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a124b86-a24c-4178-9b37-778b4ff52f6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:51.160619 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a124b86-a24c-4178-9b37-778b4ff52f6e None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:51.166372 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97cebd-b8ed-4265-a151-07f69f99dfb5 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:51.265749 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 8/16] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:21:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 16:21:51.268605 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97cebd-b8ed-4265-a151-07f69f99dfb5 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:51.271160 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a124b86-a24c-4178-9b37-778b4ff52f6e None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:51.273844 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:51.373909 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a124b86-a24c-4178-9b37-778b4ff52f6e None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:51.409381 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f901d-e00b-4635-a3eb-5a52b8ac873f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:51.410028 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f901d-e00b-4635-a3eb-5a52b8ac873f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:51.410314 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f901d-e00b-4635-a3eb-5a52b8ac873f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:51.410596 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f901d-e00b-4635-a3eb-5a52b8ac873f None admin] PATH_INFO: `/v3/users/2682f5470265422cbaaa91dfd16e8ea6` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:51.411233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c62b45-d531-441c-98b7-47522beae34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:51.411783 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f901d-e00b-4635-a3eb-5a52b8ac873f None admin] RBAC: Authorizing `identity:get_user(user_id=2682f5470265422cbaaa91dfd16e8ea6)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:51.412096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c62b45-d531-441c-98b7-47522beae34c None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:51.412389 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c62b45-d531-441c-98b7-47522beae34c None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:51.412665 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c62b45-d531-441c-98b7-47522beae34c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:51.413269 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f901d-e00b-4635-a3eb-5a52b8ac873f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:51.414870 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 9/17] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:21:51 2022] GET /identity/v3/users/2682f5470265422cbaaa91dfd16e8ea6 => generated 247 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:21:51.420192 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f901d-e00b-4635-a3eb-5a52b8ac873f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:51.435522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c62b45-d531-441c-98b7-47522beae34c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:51.436078 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:51.437149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c62b45-d531-441c-98b7-47522beae34c None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:51.456384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-64c62b45-d531-441c-98b7-47522beae34c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:21:51.456384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:21:51.456384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:21:51.456384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:21:51.456384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:21:51.456384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:21:51.456384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:21:51.456384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:21:51.456384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:21:51.456384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:21:51.456384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:21:51.456384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:21:51.456384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:21:51.456384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:21:51.456384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:21:51.457406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:21:51.457406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:21:51.457406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:21:51.457406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:21:51.457406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:21:51.457406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:21:51.457406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:21:51.457406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:21:51.457406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:21:51.457406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:21:51.457406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:21:51.457406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:21:51.457406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:21:51.457406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:21:51.457406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:21:51.457406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:21:51.457406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:21:51.458369 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:21:51.458369 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:21:51.458369 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:21:51.458369 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:21:51.458369 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 9/18] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:21:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:21:51.463506 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c62b45-d531-441c-98b7-47522beae34c None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:51.519522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:51.524067 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f901d-e00b-4635-a3eb-5a52b8ac873f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:51.557220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec17cc-5e91-483c-936b-1883f463ebe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:51.557863 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec17cc-5e91-483c-936b-1883f463ebe4 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:51.558148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec17cc-5e91-483c-936b-1883f463ebe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:51.558428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec17cc-5e91-483c-936b-1883f463ebe4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:51.559603 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec17cc-5e91-483c-936b-1883f463ebe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:51.561144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec17cc-5e91-483c-936b-1883f463ebe4 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:51.562951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 10/19] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:21:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:21:51.563070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c62b45-d531-441c-98b7-47522beae34c None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:51.568422 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec17cc-5e91-483c-936b-1883f463ebe4 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:51.601023 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b23b6e-71b8-48e7-9fb4-83b7610c2c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:51.601658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b23b6e-71b8-48e7-9fb4-83b7610c2c71 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:51.601940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b23b6e-71b8-48e7-9fb4-83b7610c2c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:51.602210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b23b6e-71b8-48e7-9fb4-83b7610c2c71 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:51.602907 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b23b6e-71b8-48e7-9fb4-83b7610c2c71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:21:51.603207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b23b6e-71b8-48e7-9fb4-83b7610c2c71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:51.604578 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b23b6e-71b8-48e7-9fb4-83b7610c2c71 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:51.621821 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 10/20] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:21:51.650584 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:51.671533 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec17cc-5e91-483c-936b-1883f463ebe4 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:51.710611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd85778-3272-43aa-b638-19d9b696f53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:51.711257 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd85778-3272-43aa-b638-19d9b696f53a None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:51.711564 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd85778-3272-43aa-b638-19d9b696f53a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:51.712145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd85778-3272-43aa-b638-19d9b696f53a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:51.712617 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd85778-3272-43aa-b638-19d9b696f53a None admin] RBAC: Adding query filter params (role.id=ca37e43082fa40c6a88b12680a0ca94b, scope.domain.id=default, user.id=2682f5470265422cbaaa91dfd16e8ea6) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:21:51.713246 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd85778-3272-43aa-b638-19d9b696f53a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:51.714720 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd85778-3272-43aa-b638-19d9b696f53a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:51.736938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 11/21] 10.222.0.74 () {56 vars in 1322 bytes} [Wed Jul 27 16:21:51 2022] GET /identity/v3/role_assignments?user.id=2682f5470265422cbaaa91dfd16e8ea6&scope.domain.id=default&role.id=ca37e43082fa40c6a88b12680a0ca94b => generated 233 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:21:53.997757 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ae7d6-446a-4206-aaef-f634cab86775 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:53.998347 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ae7d6-446a-4206-aaef-f634cab86775 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:53.998426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ae7d6-446a-4206-aaef-f634cab86775 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:53.999929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 11/22] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:21:54.005325 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4a39f-71aa-4967-b551-98792862667d None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:54.005727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4a39f-71aa-4967-b551-98792862667d None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:54.005926 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4a39f-71aa-4967-b551-98792862667d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:54.030646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-94a4a39f-71aa-4967-b551-98792862667d None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:21:54.031855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-94a4a39f-71aa-4967-b551-98792862667d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:21:54.122735 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 12/23] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:21:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:21:54.160597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ae7d6-446a-4206-aaef-f634cab86775 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:54.238088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cf2a6-fef1-46e2-b74f-558414bc959a None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:54.238717 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cf2a6-fef1-46e2-b74f-558414bc959a None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:54.239217 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cf2a6-fef1-46e2-b74f-558414bc959a None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:54.244668 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 13/24] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:21:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:21:54.246071 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2cd47-0109-40b6-80ae-79e0d4de845a None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:54.246592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2cd47-0109-40b6-80ae-79e0d4de845a None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:54.247074 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2cd47-0109-40b6-80ae-79e0d4de845a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:54.281980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-12a2cd47-0109-40b6-80ae-79e0d4de845a None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:21:54.283488 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-12a2cd47-0109-40b6-80ae-79e0d4de845a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:21:54.318722 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ae7d6-446a-4206-aaef-f634cab86775 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:54.349277 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:54.370298 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36757f6b-cfe5-458b-a349-c0369622611d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:54.370938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36757f6b-cfe5-458b-a349-c0369622611d None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:54.371222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36757f6b-cfe5-458b-a349-c0369622611d None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:54.371500 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36757f6b-cfe5-458b-a349-c0369622611d None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:54.372242 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36757f6b-cfe5-458b-a349-c0369622611d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:54.377712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36757f6b-cfe5-458b-a349-c0369622611d None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:54.438595 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 14/25] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:21:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:21:54.472990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2cd47-0109-40b6-80ae-79e0d4de845a None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:54.476720 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:54.643203 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 12/26] 10.222.0.74 () {60 vars in 1131 bytes} [Wed Jul 27 16:21:54 2022] POST /identity/v3/roles => generated 219 bytes in 483 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:21:54.644399 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e33a5a-9509-4228-88bb-26dc0677211a None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:54.644655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e33a5a-9509-4228-88bb-26dc0677211a None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:54.644958 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e33a5a-9509-4228-88bb-26dc0677211a None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:54.646178 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 13/27] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:21:54.654442 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ae127-a529-4fb2-a394-7906008ff5e9 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:54.655041 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ae127-a529-4fb2-a394-7906008ff5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:54.655469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ae127-a529-4fb2-a394-7906008ff5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:54.696082 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-bf4ae127-a529-4fb2-a394-7906008ff5e9 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:21:54.699421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-bf4ae127-a529-4fb2-a394-7906008ff5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:21:54.704314 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2cd47-0109-40b6-80ae-79e0d4de845a None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:54.796128 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651cfe6-4a2d-457d-bdea-f6cc856858c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:54.798236 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651cfe6-4a2d-457d-bdea-f6cc856858c5 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:54.799039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651cfe6-4a2d-457d-bdea-f6cc856858c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:54.799472 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651cfe6-4a2d-457d-bdea-f6cc856858c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:54.822426 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651cfe6-4a2d-457d-bdea-f6cc856858c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:54.824472 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651cfe6-4a2d-457d-bdea-f6cc856858c5 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:54.844197 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-d651cfe6-4a2d-457d-bdea-f6cc856858c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:21:54.844197 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:21:54.844197 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:21:54.844197 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:21:54.844197 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:21:54.844197 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:21:54.844197 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:21:54.844197 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:21:54.844197 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:21:54.844197 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:21:54.844197 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:21:54.844197 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:21:54.844197 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:21:54.844197 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:21:54.844197 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:21:54.845093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:21:54.845093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:21:54.845093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:21:54.845093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:21:54.845093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:21:54.845093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:21:54.845093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:21:54.845093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:21:54.845093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:21:54.845093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:21:54.845093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:21:54.845093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:21:54.845093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:21:54.845093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:21:54.845093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:21:54.845093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:21:54.845093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:21:54.845880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:21:54.845880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:21:54.845880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:21:54.845880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:21:54.847824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 15/28] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:21:54.849274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601e924-7bc1-415e-8a21-a55d408d4544 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:54.849756 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601e924-7bc1-415e-8a21-a55d408d4544 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:54.850249 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601e924-7bc1-415e-8a21-a55d408d4544 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:54.856161 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 14/29] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:21:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:21:54.856295 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 16/30] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:21:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:21:54.858680 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ae127-a529-4fb2-a394-7906008ff5e9 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:54.860686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198088d-c070-4637-8794-7dbeeb0d36ce None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:54.861005 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198088d-c070-4637-8794-7dbeeb0d36ce None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:54.861285 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198088d-c070-4637-8794-7dbeeb0d36ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:54.886668 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-a198088d-c070-4637-8794-7dbeeb0d36ce None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:21:54.888048 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-a198088d-c070-4637-8794-7dbeeb0d36ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:21:54.965270 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:54.986463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 17/31] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:21:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:21:54.989228 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198088d-c070-4637-8794-7dbeeb0d36ce None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:54.999595 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ae127-a529-4fb2-a394-7906008ff5e9 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:55.023054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:55.059317 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4eadb9-c9d4-48e5-bb66-e7c88313a58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:55.059987 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4eadb9-c9d4-48e5-bb66-e7c88313a58e None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:55.060273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4eadb9-c9d4-48e5-bb66-e7c88313a58e None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:55.060555 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4eadb9-c9d4-48e5-bb66-e7c88313a58e None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:55.061852 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4eadb9-c9d4-48e5-bb66-e7c88313a58e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:21:55.062163 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4eadb9-c9d4-48e5-bb66-e7c88313a58e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:55.064174 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4eadb9-c9d4-48e5-bb66-e7c88313a58e None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:55.079180 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 15/32] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:21:55.083944 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4eadb9-c9d4-48e5-bb66-e7c88313a58e None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:55.107878 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198088d-c070-4637-8794-7dbeeb0d36ce None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:55.142735 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2deaff-876c-448a-9648-3464a84ed713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:55.143389 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2deaff-876c-448a-9648-3464a84ed713 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:55.143677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2deaff-876c-448a-9648-3464a84ed713 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:55.143954 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2deaff-876c-448a-9648-3464a84ed713 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:55.154651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2deaff-876c-448a-9648-3464a84ed713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:21:55.159726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-4b2deaff-876c-448a-9648-3464a84ed713 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:21:55.159726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:21:55.159726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:21:55.159726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:21:55.159726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:21:55.159726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:21:55.159726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:21:55.159726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:21:55.159726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:21:55.159726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:21:55.159726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:21:55.159726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:21:55.159726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:21:55.159726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:21:55.159726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:21:55.159726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:21:55.160659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:21:55.160659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:21:55.160659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:21:55.160659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:21:55.160659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:21:55.160659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:21:55.160659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:21:55.160659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:21:55.160659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:21:55.160659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:21:55.160659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:21:55.160659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:21:55.160659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:21:55.160659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:21:55.160659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:21:55.160659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:21:55.161450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:21:55.161450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:21:55.161450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:21:55.161450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:21:55.161450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:21:55.161450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:21:55.161450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:21:55.161450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:21:55.161450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:21:55.161450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:21:55.161450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:21:55.161450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:21:55.161450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 18/33] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:21:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:21:55.162092 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2deaff-876c-448a-9648-3464a84ed713 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:55.192806 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4eadb9-c9d4-48e5-bb66-e7c88313a58e None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:55.213481 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:55.230178 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fb23b-d76f-47f7-b5aa-7563943adf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:55.231170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fb23b-d76f-47f7-b5aa-7563943adf10 None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:55.231599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fb23b-d76f-47f7-b5aa-7563943adf10 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:55.232475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fb23b-d76f-47f7-b5aa-7563943adf10 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:55.234176 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fb23b-d76f-47f7-b5aa-7563943adf10 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:55.235880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fb23b-d76f-47f7-b5aa-7563943adf10 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:55.258614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 16/34] 10.222.0.74 () {60 vars in 1131 bytes} [Wed Jul 27 16:21:55 2022] POST /identity/v3/roles => generated 225 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:21:55.259845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fb23b-d76f-47f7-b5aa-7563943adf10 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:55.282682 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2deaff-876c-448a-9648-3464a84ed713 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:55.321450 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:55.362777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5da1da-24cc-4610-a7e4-6231d5c2bba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:55.363536 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5da1da-24cc-4610-a7e4-6231d5c2bba0 None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:55.363904 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5da1da-24cc-4610-a7e4-6231d5c2bba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:55.364187 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5da1da-24cc-4610-a7e4-6231d5c2bba0 None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:55.364913 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5da1da-24cc-4610-a7e4-6231d5c2bba0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:55.366391 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5da1da-24cc-4610-a7e4-6231d5c2bba0 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:55.442694 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fb23b-d76f-47f7-b5aa-7563943adf10 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:55.485541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257720b8-2d8c-47f4-9459-1113fad33b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:55.486096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257720b8-2d8c-47f4-9459-1113fad33b96 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:55.486383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257720b8-2d8c-47f4-9459-1113fad33b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:55.486661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257720b8-2d8c-47f4-9459-1113fad33b96 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:55.487382 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257720b8-2d8c-47f4-9459-1113fad33b96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:21:55.487692 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257720b8-2d8c-47f4-9459-1113fad33b96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:55.490469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257720b8-2d8c-47f4-9459-1113fad33b96 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:55.502660 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 17/35] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:21:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:21:55.503990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b2225-d258-407a-b92a-5d96544f4094 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:55.504351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b2225-d258-407a-b92a-5d96544f4094 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:55.504720 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b2225-d258-407a-b92a-5d96544f4094 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:55.506223 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 18/36] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:21:55.510294 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b2225-d258-407a-b92a-5d96544f4094 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:55.521091 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 19/37] 10.222.0.74 () {60 vars in 1141 bytes} [Wed Jul 27 16:21:55 2022] POST /identity/v3/projects => generated 301 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:21:55.522417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ed277-0efb-4d9b-a58b-f55d0bef4049 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:55.522717 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ed277-0efb-4d9b-a58b-f55d0bef4049 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:55.523007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ed277-0efb-4d9b-a58b-f55d0bef4049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:55.543834 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-d32ed277-0efb-4d9b-a58b-f55d0bef4049 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:21:55.545682 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d32ed277-0efb-4d9b-a58b-f55d0bef4049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:21:55.619291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:55.631091 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 20/38] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:21:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:21:55.631091 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39d7c6-260b-4167-81ad-4a834f829440 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:55.631091 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39d7c6-260b-4167-81ad-4a834f829440 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:55.631091 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39d7c6-260b-4167-81ad-4a834f829440 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:55.631091 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 21/39] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:21:55.632697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441e110-c0b3-4200-abc9-48422673a396 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:55.633024 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441e110-c0b3-4200-abc9-48422673a396 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:55.633312 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441e110-c0b3-4200-abc9-48422673a396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:55.643856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b2225-d258-407a-b92a-5d96544f4094 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:55.653761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-3441e110-c0b3-4200-abc9-48422673a396 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:21:55.655867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-3441e110-c0b3-4200-abc9-48422673a396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:21:55.681400 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:55.684474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4882d8-49fd-47b5-b851-f90a4eb27255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:55.685183 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4882d8-49fd-47b5-b851-f90a4eb27255 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:55.685464 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4882d8-49fd-47b5-b851-f90a4eb27255 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:55.686277 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4882d8-49fd-47b5-b851-f90a4eb27255 None admin] PATH_INFO: `/v3/users/2682f5470265422cbaaa91dfd16e8ea6` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:55.688360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4882d8-49fd-47b5-b851-f90a4eb27255 None admin] RBAC: Authorizing `identity:get_user(user_id=2682f5470265422cbaaa91dfd16e8ea6)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:55.690098 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4882d8-49fd-47b5-b851-f90a4eb27255 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:55.691724 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 19/40] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:21:55 2022] GET /identity/v3/users/2682f5470265422cbaaa91dfd16e8ea6 => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:21:55.692259 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4882d8-49fd-47b5-b851-f90a4eb27255 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:55.743345 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 22/41] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:21:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:21:55.744053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441e110-c0b3-4200-abc9-48422673a396 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:55.815012 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4882d8-49fd-47b5-b851-f90a4eb27255 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:55.844741 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441e110-c0b3-4200-abc9-48422673a396 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:55.848122 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9c9ee-cbb8-4479-bccc-f4beecfc9a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:55.848746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9c9ee-cbb8-4479-bccc-f4beecfc9a36 None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:55.849062 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9c9ee-cbb8-4479-bccc-f4beecfc9a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:55.849339 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9c9ee-cbb8-4479-bccc-f4beecfc9a36 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:55.850047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9c9ee-cbb8-4479-bccc-f4beecfc9a36 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:55.851406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9c9ee-cbb8-4479-bccc-f4beecfc9a36 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:55.866651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 20/42] 10.222.0.74 () {60 vars in 1131 bytes} [Wed Jul 27 16:21:55 2022] POST /identity/v3/roles => generated 223 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:21:55.866651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9c9ee-cbb8-4479-bccc-f4beecfc9a36 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:55.890741 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:55.897627 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bc8c5-a389-40d4-b414-88159042e6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:55.898363 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bc8c5-a389-40d4-b414-88159042e6d5 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:55.898649 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bc8c5-a389-40d4-b414-88159042e6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:55.898931 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bc8c5-a389-40d4-b414-88159042e6d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:55.900536 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0bc8c5-a389-40d4-b414-88159042e6d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:55.902491 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0bc8c5-a389-40d4-b414-88159042e6d5 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:55.908816 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 23/43] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:21:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:21:55.910012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bc8c5-a389-40d4-b414-88159042e6d5 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:55.940713 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:55.996934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9c9ee-cbb8-4479-bccc-f4beecfc9a36 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:56.025188 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bc8c5-a389-40d4-b414-88159042e6d5 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:56.041311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ce17f-3510-48ee-9d63-16c86dce18ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:56.041311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ce17f-3510-48ee-9d63-16c86dce18ec None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:56.041311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ce17f-3510-48ee-9d63-16c86dce18ec None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:56.041311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ce17f-3510-48ee-9d63-16c86dce18ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:56.042768 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ce17f-3510-48ee-9d63-16c86dce18ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:56.044482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ce17f-3510-48ee-9d63-16c86dce18ec None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:56.045472 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 21/44] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:21:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:21:56.051469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ce17f-3510-48ee-9d63-16c86dce18ec None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:56.063458 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742782d-7c7e-42cc-ba8d-9d4e63034c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:56.064131 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742782d-7c7e-42cc-ba8d-9d4e63034c34 None admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:56.064420 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742782d-7c7e-42cc-ba8d-9d4e63034c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:56.064705 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742782d-7c7e-42cc-ba8d-9d4e63034c34 None admin] PATH_INFO: `/v3/domains/default/users/2682f5470265422cbaaa91dfd16e8ea6/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:56.066796 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742782d-7c7e-42cc-ba8d-9d4e63034c34 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=2682f5470265422cbaaa91dfd16e8ea6, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:56.068269 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742782d-7c7e-42cc-ba8d-9d4e63034c34 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:56.084771 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 24/45] 10.222.0.74 () {58 vars in 1361 bytes} [Wed Jul 27 16:21:55 2022] PUT /identity/v3/domains/default/users/2682f5470265422cbaaa91dfd16e8ea6/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:21:56.143906 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:56.162329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ce17f-3510-48ee-9d63-16c86dce18ec None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:56.198406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef25fd-60cd-4401-a1dd-c56e67c61a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:56.199047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef25fd-60cd-4401-a1dd-c56e67c61a5e None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:56.199330 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef25fd-60cd-4401-a1dd-c56e67c61a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:56.199605 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef25fd-60cd-4401-a1dd-c56e67c61a5e None admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:56.200549 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef25fd-60cd-4401-a1dd-c56e67c61a5e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:56.202084 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef25fd-60cd-4401-a1dd-c56e67c61a5e None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:56.232050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 22/46] 10.222.0.74 () {60 vars in 1141 bytes} [Wed Jul 27 16:21:56 2022] POST /identity/v3/projects => generated 312 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:21:58.186599 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426d5cf-3ad6-4878-af1e-08d6333e5a0e None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:58.187009 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426d5cf-3ad6-4878-af1e-08d6333e5a0e None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:58.187174 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426d5cf-3ad6-4878-af1e-08d6333e5a0e None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:58.188551 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 25/47] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:21:58.193910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a37c06-7fd8-4f46-b1b8-eb7530928ef3 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:58.194304 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a37c06-7fd8-4f46-b1b8-eb7530928ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:58.194480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a37c06-7fd8-4f46-b1b8-eb7530928ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:58.212739 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-95a37c06-7fd8-4f46-b1b8-eb7530928ef3 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:21:58.213956 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-95a37c06-7fd8-4f46-b1b8-eb7530928ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:21:58.251286 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b427cb-e02e-4296-9ac2-57d5fb3054bf None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:58.251569 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b427cb-e02e-4296-9ac2-57d5fb3054bf None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:58.251845 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b427cb-e02e-4296-9ac2-57d5fb3054bf None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:58.253233 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 26/48] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:21:58.258092 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8aaa5-95ca-48f6-a828-8827fb86dc2d None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:58.258387 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8aaa5-95ca-48f6-a828-8827fb86dc2d None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:58.258691 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8aaa5-95ca-48f6-a828-8827fb86dc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:58.276725 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-97e8aaa5-95ca-48f6-a828-8827fb86dc2d None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:21:58.277965 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-97e8aaa5-95ca-48f6-a828-8827fb86dc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:21:58.280427 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 23/49] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:21:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:21:58.306472 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a37c06-7fd8-4f46-b1b8-eb7530928ef3 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:58.346080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 27/50] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:21:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:21:58.368251 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8aaa5-95ca-48f6-a828-8827fb86dc2d None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:58.369867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:58.385197 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:58.406776 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a37c06-7fd8-4f46-b1b8-eb7530928ef3 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:58.439696 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937a6a7-fa80-41da-8397-3d2eced34bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:58.440270 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937a6a7-fa80-41da-8397-3d2eced34bdf None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:58.440553 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937a6a7-fa80-41da-8397-3d2eced34bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:58.440829 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937a6a7-fa80-41da-8397-3d2eced34bdf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:58.452512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937a6a7-fa80-41da-8397-3d2eced34bdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:21:58.459129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-c937a6a7-fa80-41da-8397-3d2eced34bdf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:21:58.459129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:21:58.459129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:21:58.459129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:21:58.459129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:21:58.459129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:21:58.459129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:21:58.459129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:21:58.459129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:21:58.459129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:21:58.459129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:21:58.459129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:21:58.459129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:21:58.459129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:21:58.459129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:21:58.459129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:21:58.460051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:21:58.460051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:21:58.460051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:21:58.460051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:21:58.460051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:21:58.460051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:21:58.460051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:21:58.460051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:21:58.460051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:21:58.460051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:21:58.460051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:21:58.460051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:21:58.460051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:21:58.460051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:21:58.460051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:21:58.460051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:21:58.460914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:21:58.460914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:21:58.460914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:21:58.460914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:21:58.460914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:21:58.460914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:21:58.460914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:21:58.460914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:21:58.460914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:21:58.460914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:21:58.460914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:21:58.460914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:21:58.460914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 24/51] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:21:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:21:58.465936 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937a6a7-fa80-41da-8397-3d2eced34bdf None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:58.467067 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8aaa5-95ca-48f6-a828-8827fb86dc2d None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:58.501219 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50607b8-7f04-4da5-82f2-d29263dce5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:58.501869 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50607b8-7f04-4da5-82f2-d29263dce5cc None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:58.502155 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50607b8-7f04-4da5-82f2-d29263dce5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:58.502433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50607b8-7f04-4da5-82f2-d29263dce5cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:58.503588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50607b8-7f04-4da5-82f2-d29263dce5cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:58.505083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50607b8-7f04-4da5-82f2-d29263dce5cc None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:58.506677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 28/52] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:21:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:21:58.512087 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50607b8-7f04-4da5-82f2-d29263dce5cc None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:58.565147 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937a6a7-fa80-41da-8397-3d2eced34bdf None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:58.590469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:58.594274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:58.600368 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a292451-ca0b-48fa-9372-a422aaec8b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:58.601019 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a292451-ca0b-48fa-9372-a422aaec8b0e None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:58.601302 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a292451-ca0b-48fa-9372-a422aaec8b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:58.601585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a292451-ca0b-48fa-9372-a422aaec8b0e None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:58.602297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a292451-ca0b-48fa-9372-a422aaec8b0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:21:58.602602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a292451-ca0b-48fa-9372-a422aaec8b0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:58.603981 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a292451-ca0b-48fa-9372-a422aaec8b0e None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:58.609832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50607b8-7f04-4da5-82f2-d29263dce5cc None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:58.612147 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 25/53] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:21:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:21:58.617418 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a292451-ca0b-48fa-9372-a422aaec8b0e None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:58.643182 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda92ffe-281e-4f8f-89fb-5332e703f848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:58.643807 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda92ffe-281e-4f8f-89fb-5332e703f848 None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:58.644091 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda92ffe-281e-4f8f-89fb-5332e703f848 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:58.644369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda92ffe-281e-4f8f-89fb-5332e703f848 None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:58.645074 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda92ffe-281e-4f8f-89fb-5332e703f848 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:58.646522 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda92ffe-281e-4f8f-89fb-5332e703f848 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:58.673929 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 29/54] 10.222.0.74 () {60 vars in 1140 bytes} [Wed Jul 27 16:21:58 2022] POST /identity/v3/projects => generated 298 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:21:58.719984 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a292451-ca0b-48fa-9372-a422aaec8b0e None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:58.756140 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6eaa9-b8f6-4959-96a2-0f9a707d7009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:58.756805 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6eaa9-b8f6-4959-96a2-0f9a707d7009 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:58.757129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6eaa9-b8f6-4959-96a2-0f9a707d7009 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:58.757408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6eaa9-b8f6-4959-96a2-0f9a707d7009 None admin] PATH_INFO: `/v3/users/2682f5470265422cbaaa91dfd16e8ea6` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:58.758654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6eaa9-b8f6-4959-96a2-0f9a707d7009 None admin] RBAC: Authorizing `identity:get_user(user_id=2682f5470265422cbaaa91dfd16e8ea6)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:58.760184 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6eaa9-b8f6-4959-96a2-0f9a707d7009 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:58.761867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 26/55] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:21:58 2022] GET /identity/v3/users/2682f5470265422cbaaa91dfd16e8ea6 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:21:58.767586 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda92ffe-281e-4f8f-89fb-5332e703f848 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:58.872866 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda92ffe-281e-4f8f-89fb-5332e703f848 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:58.907762 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593e304-20c1-4f9d-97bd-7134069fc84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:58.908412 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593e304-20c1-4f9d-97bd-7134069fc84f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:58.908701 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593e304-20c1-4f9d-97bd-7134069fc84f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:58.909000 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593e304-20c1-4f9d-97bd-7134069fc84f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:58.910143 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8593e304-20c1-4f9d-97bd-7134069fc84f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:58.911584 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8593e304-20c1-4f9d-97bd-7134069fc84f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:58.913289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 30/56] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:21:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:21:58.918975 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6eaa9-b8f6-4959-96a2-0f9a707d7009 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:21:58.974880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:21:59.019944 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6eaa9-b8f6-4959-96a2-0f9a707d7009 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:21:59.052944 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e061a-72dc-47d4-876b-d1a985cacb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:21:59.053571 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e061a-72dc-47d4-876b-d1a985cacb25 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:21:59.053857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e061a-72dc-47d4-876b-d1a985cacb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:21:59.054136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e061a-72dc-47d4-876b-d1a985cacb25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:21:59.054925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e061a-72dc-47d4-876b-d1a985cacb25 None admin] RBAC: Adding query filter params (role.id=ca37e43082fa40c6a88b12680a0ca94b, scope.domain.id=default, user.id=2682f5470265422cbaaa91dfd16e8ea6) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:21:59.055236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e061a-72dc-47d4-876b-d1a985cacb25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:21:59.056683 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e061a-72dc-47d4-876b-d1a985cacb25 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:21:59.065807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 27/57] 10.222.0.74 () {56 vars in 1322 bytes} [Wed Jul 27 16:21:58 2022] GET /identity/v3/role_assignments?user.id=2682f5470265422cbaaa91dfd16e8ea6&scope.domain.id=default&role.id=ca37e43082fa40c6a88b12680a0ca94b => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:00.667141 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51eb62a-9f66-4c75-bf63-e50bc12b7d4c None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:00.667650 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51eb62a-9f66-4c75-bf63-e50bc12b7d4c None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:00.667732 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51eb62a-9f66-4c75-bf63-e50bc12b7d4c None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:00.669087 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 31/58] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:00.674663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa24e8-9dc1-4615-aad4-76f6d8d3736e None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:00.675078 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa24e8-9dc1-4615-aad4-76f6d8d3736e None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:00.675235 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa24e8-9dc1-4615-aad4-76f6d8d3736e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:00.693642 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-26aa24e8-9dc1-4615-aad4-76f6d8d3736e None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:00.694833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-26aa24e8-9dc1-4615-aad4-76f6d8d3736e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:00.760228 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 28/59] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:00.783609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51eb62a-9f66-4c75-bf63-e50bc12b7d4c None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:00.787382 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:00.881816 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51eb62a-9f66-4c75-bf63-e50bc12b7d4c None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:00.913481 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b411d-f368-44a2-8c88-21778c94934c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:00.914122 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b411d-f368-44a2-8c88-21778c94934c None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:00.914411 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b411d-f368-44a2-8c88-21778c94934c None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:00.914689 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b411d-f368-44a2-8c88-21778c94934c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:00.915810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72b411d-f368-44a2-8c88-21778c94934c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:00.917335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72b411d-f368-44a2-8c88-21778c94934c None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:00.918857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 32/60] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:22:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:00.924757 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa24e8-9dc1-4615-aad4-76f6d8d3736e None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:00.973864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:01.037249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa24e8-9dc1-4615-aad4-76f6d8d3736e None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:01.077326 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8358817-3446-4b93-b683-93ae3343ded7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:01.077978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8358817-3446-4b93-b683-93ae3343ded7 None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:01.078275 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8358817-3446-4b93-b683-93ae3343ded7 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:01.078554 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8358817-3446-4b93-b683-93ae3343ded7 None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:01.079246 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8358817-3446-4b93-b683-93ae3343ded7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:01.081197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8358817-3446-4b93-b683-93ae3343ded7 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:01.110885 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 29/61] 10.222.0.74 () {60 vars in 1132 bytes} [Wed Jul 27 16:22:00 2022] POST /identity/v3/users => generated 275 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:22:04.210119 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2eaff-5679-47f5-9f9b-f1e3249c808b None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:04.210636 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2eaff-5679-47f5-9f9b-f1e3249c808b None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:04.210725 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2eaff-5679-47f5-9f9b-f1e3249c808b None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:04.212034 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 33/62] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:04.217251 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932a1f3-8ced-4377-ab3e-8942fdf3d6bc None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:04.217669 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932a1f3-8ced-4377-ab3e-8942fdf3d6bc None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:04.217840 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932a1f3-8ced-4377-ab3e-8942fdf3d6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:04.232767 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a7021-92fd-4757-88b7-7a027be67b8c None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:04.233093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a7021-92fd-4757-88b7-7a027be67b8c None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:04.233374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a7021-92fd-4757-88b7-7a027be67b8c None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:04.234719 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 34/63] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:22:04.240066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452036ef-415c-4dfd-9b8f-d198d53856b6 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:04.240351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452036ef-415c-4dfd-9b8f-d198d53856b6 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:04.240628 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452036ef-415c-4dfd-9b8f-d198d53856b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:04.256909 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-3932a1f3-8ced-4377-ab3e-8942fdf3d6bc None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:04.258229 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-3932a1f3-8ced-4377-ab3e-8942fdf3d6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:04.271745 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-452036ef-415c-4dfd-9b8f-d198d53856b6 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:04.272964 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-452036ef-415c-4dfd-9b8f-d198d53856b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:04.350749 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 35/64] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:04.378747 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452036ef-415c-4dfd-9b8f-d198d53856b6 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:04.380907 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:04.391491 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 30/65] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:04.417304 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932a1f3-8ced-4377-ab3e-8942fdf3d6bc None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:04.489296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:04.564114 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452036ef-415c-4dfd-9b8f-d198d53856b6 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:04.605071 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d095ada-e784-4e5b-bdf4-2023a6d3f070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:04.605730 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d095ada-e784-4e5b-bdf4-2023a6d3f070 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:04.606022 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d095ada-e784-4e5b-bdf4-2023a6d3f070 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:04.606306 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d095ada-e784-4e5b-bdf4-2023a6d3f070 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:04.606913 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932a1f3-8ced-4377-ab3e-8942fdf3d6bc None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:04.617052 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d095ada-e784-4e5b-bdf4-2023a6d3f070 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:04.622253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-5d095ada-e784-4e5b-bdf4-2023a6d3f070 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:04.622253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:04.622253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:04.622253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:04.622253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:04.622253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:04.622253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:04.622253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:04.622253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:04.622253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:04.622253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:04.622253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:04.622253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:04.622253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:04.622253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:04.622253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:04.623846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:04.623846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:04.623846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:04.623846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:04.623846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:04.623846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:04.623846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:04.623846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:04.623846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:04.623846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:04.623846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:04.623846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:04.623846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:04.623846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:04.623846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:04.623846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:04.624876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:04.624876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:04.624876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:04.624876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:04.624876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:04.624876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:04.624876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:04.624876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:04.624876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:04.624876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:04.624876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:04.624876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:04.626154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 36/66] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:04.631906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d095ada-e784-4e5b-bdf4-2023a6d3f070 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:04.679612 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e995b-f4d1-42cf-baf6-a624cca6f53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:04.680393 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e995b-f4d1-42cf-baf6-a624cca6f53c None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:04.680750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e995b-f4d1-42cf-baf6-a624cca6f53c None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:04.681131 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e995b-f4d1-42cf-baf6-a624cca6f53c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:04.692902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e995b-f4d1-42cf-baf6-a624cca6f53c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:04.696037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:04.698318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-1b7e995b-f4d1-42cf-baf6-a624cca6f53c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:22:04.698318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:04.698318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:04.698318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:04.698318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:04.698318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:04.698318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:04.698318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:04.698318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:04.698318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:04.698318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:04.698318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:04.698318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:04.698318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:04.698318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:04.698318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:04.699182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:04.699182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:04.699182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:04.699182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:04.699182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:04.699182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:04.699182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:04.699182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:04.699182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:04.699182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:04.699182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:04.699182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:04.699182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:04.699182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:04.699182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:04.699182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:04.699929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:04.699929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:04.699929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:04.699929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:04.699929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:04.699929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:04.699929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:04.699929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:04.699929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:04.699929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:04.699929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:22:04.699929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:04.704516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 31/67] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:22:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:22:04.706738 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb054300-d580-426c-874c-a73138de9c99 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:04.706738 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb054300-d580-426c-874c-a73138de9c99 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:04.706933 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb054300-d580-426c-874c-a73138de9c99 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:04.708346 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 32/68] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:04.713233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112b760-41df-4c28-85b3-ebadb3e8b624 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:04.713526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112b760-41df-4c28-85b3-ebadb3e8b624 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:04.713806 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112b760-41df-4c28-85b3-ebadb3e8b624 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:04.719509 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 33/69] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:04.719509 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112b760-41df-4c28-85b3-ebadb3e8b624 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:04.744580 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d095ada-e784-4e5b-bdf4-2023a6d3f070 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:04.781809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c7779-4217-41b1-8afa-a6a451bfff9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:04.782439 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c7779-4217-41b1-8afa-a6a451bfff9b None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:04.782729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c7779-4217-41b1-8afa-a6a451bfff9b None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:04.783014 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c7779-4217-41b1-8afa-a6a451bfff9b None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:04.783733 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c7779-4217-41b1-8afa-a6a451bfff9b None admin] RBAC: Adding query filter params (name=member) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:04.784042 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c7779-4217-41b1-8afa-a6a451bfff9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:04.785486 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c7779-4217-41b1-8afa-a6a451bfff9b None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:04.798519 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 37/70] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:04.798519 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8dabb9-8137-4a8f-997f-108c8da6c75e None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:04.798519 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8dabb9-8137-4a8f-997f-108c8da6c75e None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:04.798519 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8dabb9-8137-4a8f-997f-108c8da6c75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:04.813703 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-fc8dabb9-8137-4a8f-997f-108c8da6c75e None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:04.814907 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-fc8dabb9-8137-4a8f-997f-108c8da6c75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:04.821467 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:04.830153 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112b760-41df-4c28-85b3-ebadb3e8b624 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:04.868801 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed22d4d-04f7-4639-801f-8e484d42468d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:04.869470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed22d4d-04f7-4639-801f-8e484d42468d None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:04.869767 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed22d4d-04f7-4639-801f-8e484d42468d None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:04.870054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed22d4d-04f7-4639-801f-8e484d42468d None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:04.870846 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed22d4d-04f7-4639-801f-8e484d42468d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:04.871162 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed22d4d-04f7-4639-801f-8e484d42468d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:04.872573 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed22d4d-04f7-4639-801f-8e484d42468d None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:04.882256 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 34/71] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:04.883459 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7ad7f-a331-4688-9277-5703f6445501 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:04.883752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7ad7f-a331-4688-9277-5703f6445501 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:04.884039 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7ad7f-a331-4688-9277-5703f6445501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:04.890013 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 38/72] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:04.891068 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8dabb9-8137-4a8f-997f-108c8da6c75e None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:04.902467 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-e3f7ad7f-a331-4688-9277-5703f6445501 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:04.903618 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e3f7ad7f-a331-4688-9277-5703f6445501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:04.945790 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:04.972619 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 35/73] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:04.973909 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3e88a-72b4-4fe9-96d6-7df4df973d6e None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:04.974200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3e88a-72b4-4fe9-96d6-7df4df973d6e None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:04.974478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3e88a-72b4-4fe9-96d6-7df4df973d6e None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:04.976940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 36/74] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:04.976940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3e88a-72b4-4fe9-96d6-7df4df973d6e None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:04.995767 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8dabb9-8137-4a8f-997f-108c8da6c75e None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:05.030748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267f79a-bca3-4a5d-a2b5-2126d45f821e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:05.031479 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267f79a-bca3-4a5d-a2b5-2126d45f821e None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:05.031835 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267f79a-bca3-4a5d-a2b5-2126d45f821e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:05.032121 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267f79a-bca3-4a5d-a2b5-2126d45f821e None admin] PATH_INFO: `/v3/users/dcba272d6ea54652b4d3c43cf492f3e7` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:05.045872 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267f79a-bca3-4a5d-a2b5-2126d45f821e None admin] RBAC: Authorizing `identity:get_user(user_id=dcba272d6ea54652b4d3c43cf492f3e7)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:05.047353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267f79a-bca3-4a5d-a2b5-2126d45f821e None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:05.049011 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 39/75] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:04 2022] GET /identity/v3/users/dcba272d6ea54652b4d3c43cf492f3e7 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:05.049773 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267f79a-bca3-4a5d-a2b5-2126d45f821e None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:05.076281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:05.089219 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3e88a-72b4-4fe9-96d6-7df4df973d6e None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:05.122491 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ad116-5c72-4ab0-954e-d1864d07d94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:05.123189 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ad116-5c72-4ab0-954e-d1864d07d94a None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:05.123480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ad116-5c72-4ab0-954e-d1864d07d94a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:05.123756 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ad116-5c72-4ab0-954e-d1864d07d94a None admin] PATH_INFO: `/v3/users/2682f5470265422cbaaa91dfd16e8ea6` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:05.124950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954ad116-5c72-4ab0-954e-d1864d07d94a None admin] RBAC: Authorizing `identity:get_user(user_id=2682f5470265422cbaaa91dfd16e8ea6)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:05.126420 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954ad116-5c72-4ab0-954e-d1864d07d94a None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:05.128028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 37/76] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:04 2022] GET /identity/v3/users/2682f5470265422cbaaa91dfd16e8ea6 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:05.129142 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673be56-3a8b-4203-b431-41fc165c2085 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:05.129428 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673be56-3a8b-4203-b431-41fc165c2085 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:05.129709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673be56-3a8b-4203-b431-41fc165c2085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:05.147101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-a673be56-3a8b-4203-b431-41fc165c2085 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:05.148349 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-a673be56-3a8b-4203-b431-41fc165c2085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:05.155171 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267f79a-bca3-4a5d-a2b5-2126d45f821e None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:05.168499 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:05.189981 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04396550-86fa-4077-8450-23dd1d4027eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:05.190632 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04396550-86fa-4077-8450-23dd1d4027eb None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:05.190920 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04396550-86fa-4077-8450-23dd1d4027eb None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:05.191202 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04396550-86fa-4077-8450-23dd1d4027eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:05.203949 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04396550-86fa-4077-8450-23dd1d4027eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:05.208943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-04396550-86fa-4077-8450-23dd1d4027eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:05.208943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:05.208943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:05.208943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:05.208943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:05.208943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:05.208943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:05.208943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:05.208943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:05.208943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:05.208943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:05.208943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:05.208943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:05.208943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:05.208943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:05.208943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:05.209950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:05.209950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:05.209950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:05.209950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:05.209950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:05.209950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:05.209950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:05.209950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:05.209950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:05.209950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:05.209950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:05.209950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:05.209950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:05.209950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:05.209950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:05.209950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:05.211175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:05.211175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:05.211175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:05.211175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:05.211175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:05.211175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:05.211175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:05.211175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:05.211175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:05.211175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:05.211175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:05.211175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:05.211175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 40/77] 10.222.0.74 () {56 vars in 1114 bytes} [Wed Jul 27 16:22:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:05.211175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04396550-86fa-4077-8450-23dd1d4027eb None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:05.224744 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 38/78] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:05.231287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673be56-3a8b-4203-b431-41fc165c2085 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:05.308836 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04396550-86fa-4077-8450-23dd1d4027eb None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:05.319209 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:05.335161 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673be56-3a8b-4203-b431-41fc165c2085 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:05.343342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79c8fa-2a4d-4557-a9a4-36bf059a3c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:05.343982 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79c8fa-2a4d-4557-a9a4-36bf059a3c63 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:05.344263 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79c8fa-2a4d-4557-a9a4-36bf059a3c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:05.344538 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79c8fa-2a4d-4557-a9a4-36bf059a3c63 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:05.354594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79c8fa-2a4d-4557-a9a4-36bf059a3c63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:05.359684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-aa79c8fa-2a4d-4557-a9a4-36bf059a3c63 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:05.359684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:05.359684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:05.359684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:05.359684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:05.359684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:05.359684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:05.359684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:05.359684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:05.359684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:05.359684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:05.359684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:05.359684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:05.359684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:05.359684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:05.359684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:05.360721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:05.360721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:05.360721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:05.360721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:05.360721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:05.360721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:05.360721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:05.360721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:05.360721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:05.360721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:05.360721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:05.360721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:05.360721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:05.360721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:05.360721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:05.360721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:05.361711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:05.361711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:05.361711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:05.361711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:05.361711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:05.361711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:05.361711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:05.361711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:05.361711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:05.361711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:05.361711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:05.361711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:05.361711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 41/79] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:05.361711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79c8fa-2a4d-4557-a9a4-36bf059a3c63 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:05.373859 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16115614-ee1b-4b4c-b517-e17ae2bf8249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:05.374486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16115614-ee1b-4b4c-b517-e17ae2bf8249 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:05.374769 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16115614-ee1b-4b4c-b517-e17ae2bf8249 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:05.375045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16115614-ee1b-4b4c-b517-e17ae2bf8249 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:05.392834 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:05.441373 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16115614-ee1b-4b4c-b517-e17ae2bf8249 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:05.442843 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16115614-ee1b-4b4c-b517-e17ae2bf8249 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:05.463197 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79c8fa-2a4d-4557-a9a4-36bf059a3c63 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:05.496749 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7688858-113c-40c2-9d00-265023a0927a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:05.497413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7688858-113c-40c2-9d00-265023a0927a None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:05.497698 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7688858-113c-40c2-9d00-265023a0927a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:05.498005 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7688858-113c-40c2-9d00-265023a0927a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:05.526116 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:05.542429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 39/80] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:05.543249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16115614-ee1b-4b4c-b517-e17ae2bf8249 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:05.567233 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7688858-113c-40c2-9d00-265023a0927a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:05.568728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7688858-113c-40c2-9d00-265023a0927a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:05.598405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:05.640423 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16115614-ee1b-4b4c-b517-e17ae2bf8249 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:05.666721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 42/81] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:05.667552 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7688858-113c-40c2-9d00-265023a0927a None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:05.674155 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593fd0f-ceb5-48b3-b33e-c43dda078bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:05.674776 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593fd0f-ceb5-48b3-b33e-c43dda078bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:05.675060 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593fd0f-ceb5-48b3-b33e-c43dda078bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:05.675337 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593fd0f-ceb5-48b3-b33e-c43dda078bf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:05.676053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593fd0f-ceb5-48b3-b33e-c43dda078bf7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:05.676359 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593fd0f-ceb5-48b3-b33e-c43dda078bf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:05.677783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593fd0f-ceb5-48b3-b33e-c43dda078bf7 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:05.685821 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 40/82] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:22:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:05.686603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593fd0f-ceb5-48b3-b33e-c43dda078bf7 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:05.733745 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:05.770177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7688858-113c-40c2-9d00-265023a0927a None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:05.784686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593fd0f-ceb5-48b3-b33e-c43dda078bf7 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:05.799553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:05.804945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49afd7a2-e3c5-4d7c-a043-3c1107163b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:05.805591 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49afd7a2-e3c5-4d7c-a043-3c1107163b97 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:05.805879 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49afd7a2-e3c5-4d7c-a043-3c1107163b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:05.806160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49afd7a2-e3c5-4d7c-a043-3c1107163b97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:05.807316 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49afd7a2-e3c5-4d7c-a043-3c1107163b97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:05.808785 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49afd7a2-e3c5-4d7c-a043-3c1107163b97 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:05.810474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 43/83] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:22:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:05.811844 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49afd7a2-e3c5-4d7c-a043-3c1107163b97 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:05.822594 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da3e22-f5f5-4535-9957-83cadcfbec6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:05.823222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da3e22-f5f5-4535-9957-83cadcfbec6d None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:05.823505 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da3e22-f5f5-4535-9957-83cadcfbec6d None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:05.823783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da3e22-f5f5-4535-9957-83cadcfbec6d None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:05.824492 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da3e22-f5f5-4535-9957-83cadcfbec6d None admin] RBAC: Adding query filter params (name=member) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:05.824797 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da3e22-f5f5-4535-9957-83cadcfbec6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:05.826985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da3e22-f5f5-4535-9957-83cadcfbec6d None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:05.834931 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 41/84] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:05.835714 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da3e22-f5f5-4535-9957-83cadcfbec6d None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:05.915035 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49afd7a2-e3c5-4d7c-a043-3c1107163b97 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:05.936580 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da3e22-f5f5-4535-9957-83cadcfbec6d None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:05.946565 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:05.949587 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617a403-7569-489b-b438-4b90f2fc5b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:05.950480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617a403-7569-489b-b438-4b90f2fc5b00 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:05.950761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617a403-7569-489b-b438-4b90f2fc5b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:05.951570 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617a403-7569-489b-b438-4b90f2fc5b00 None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:05.952730 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617a403-7569-489b-b438-4b90f2fc5b00 None admin] RBAC: Authorizing `identity:get_project(project_id=5633253a5837491c995b7f11753765f8)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:05.954219 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617a403-7569-489b-b438-4b90f2fc5b00 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:05.955869 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 44/85] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:05 2022] GET /identity/v3/projects/5633253a5837491c995b7f11753765f8 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:05.956698 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617a403-7569-489b-b438-4b90f2fc5b00 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:05.972810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7b2d9-f72c-437e-b0a0-37ffe330481f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:05.973501 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7b2d9-f72c-437e-b0a0-37ffe330481f None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:05.973796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7b2d9-f72c-437e-b0a0-37ffe330481f None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:05.974081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7b2d9-f72c-437e-b0a0-37ffe330481f None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:05.975264 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7b2d9-f72c-437e-b0a0-37ffe330481f None admin] RBAC: Authorizing `identity:get_project(project_id=5633253a5837491c995b7f11753765f8)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:05.976734 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7b2d9-f72c-437e-b0a0-37ffe330481f None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:05.978438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 42/86] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:05 2022] GET /identity/v3/projects/5633253a5837491c995b7f11753765f8 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:05.979174 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7b2d9-f72c-437e-b0a0-37ffe330481f None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:06.030401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:06.063737 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617a403-7569-489b-b438-4b90f2fc5b00 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:06.079281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7b2d9-f72c-437e-b0a0-37ffe330481f None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:06.096856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605bd0d6-e704-46b7-b24c-8c574e1bde3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:06.097523 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bd0d6-e704-46b7-b24c-8c574e1bde3f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:06.097808 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bd0d6-e704-46b7-b24c-8c574e1bde3f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:06.098084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bd0d6-e704-46b7-b24c-8c574e1bde3f None admin] PATH_INFO: `/v3/users/dcba272d6ea54652b4d3c43cf492f3e7` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:06.099252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605bd0d6-e704-46b7-b24c-8c574e1bde3f None admin] RBAC: Authorizing `identity:get_user(user_id=dcba272d6ea54652b4d3c43cf492f3e7)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:06.100716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605bd0d6-e704-46b7-b24c-8c574e1bde3f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:06.102348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 45/87] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:05 2022] GET /identity/v3/users/dcba272d6ea54652b4d3c43cf492f3e7 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:06.103103 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605bd0d6-e704-46b7-b24c-8c574e1bde3f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:06.113583 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89010137-24b4-4258-947c-042d7cb52c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:06.114203 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89010137-24b4-4258-947c-042d7cb52c0e None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:06.114488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89010137-24b4-4258-947c-042d7cb52c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:06.114760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89010137-24b4-4258-947c-042d7cb52c0e None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:06.115433 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89010137-24b4-4258-947c-042d7cb52c0e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:06.116866 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89010137-24b4-4258-947c-042d7cb52c0e None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:06.137412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 43/88] 10.222.0.74 () {60 vars in 1132 bytes} [Wed Jul 27 16:22:05 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:22:06.138276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89010137-24b4-4258-947c-042d7cb52c0e None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:06.184513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:06.225128 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605bd0d6-e704-46b7-b24c-8c574e1bde3f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:06.267538 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007012b8-7e57-4c22-8931-f1ca2df0bbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:06.268279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007012b8-7e57-4c22-8931-f1ca2df0bbbc None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:06.268565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007012b8-7e57-4c22-8931-f1ca2df0bbbc None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:06.268852 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007012b8-7e57-4c22-8931-f1ca2df0bbbc None admin] PATH_INFO: `/v3/users/dcba272d6ea54652b4d3c43cf492f3e7` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:06.270104 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007012b8-7e57-4c22-8931-f1ca2df0bbbc None admin] RBAC: Authorizing `identity:get_user(user_id=dcba272d6ea54652b4d3c43cf492f3e7)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:06.271018 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89010137-24b4-4258-947c-042d7cb52c0e None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:06.271660 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007012b8-7e57-4c22-8931-f1ca2df0bbbc None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:06.278332 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 46/89] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:06 2022] GET /identity/v3/users/dcba272d6ea54652b4d3c43cf492f3e7 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:06.279697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007012b8-7e57-4c22-8931-f1ca2df0bbbc None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:06.284412 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:06.314620 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea6e20-357e-463c-be95-288fb1a5ad13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:06.315261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea6e20-357e-463c-be95-288fb1a5ad13 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:06.315553 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea6e20-357e-463c-be95-288fb1a5ad13 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:06.315832 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea6e20-357e-463c-be95-288fb1a5ad13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:06.319179 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea6e20-357e-463c-be95-288fb1a5ad13 None admin] RBAC: Adding query filter params (role.id=8de48592893c43789e1d40158943fd52, scope.project.id=5633253a5837491c995b7f11753765f8, user.id=dcba272d6ea54652b4d3c43cf492f3e7) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:06.319179 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea6e20-357e-463c-be95-288fb1a5ad13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:06.319419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea6e20-357e-463c-be95-288fb1a5ad13 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:06.327554 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 44/90] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:22:06 2022] GET /identity/v3/role_assignments?user.id=dcba272d6ea54652b4d3c43cf492f3e7&scope.project.id=5633253a5837491c995b7f11753765f8&role.id=8de48592893c43789e1d40158943fd52 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:06.328204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea6e20-357e-463c-be95-288fb1a5ad13 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:06.405211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007012b8-7e57-4c22-8931-f1ca2df0bbbc None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:06.443459 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1810a1-066c-4dc5-886f-a2112411fabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:06.444105 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1810a1-066c-4dc5-886f-a2112411fabc None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:06.444388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1810a1-066c-4dc5-886f-a2112411fabc None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:06.444660 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1810a1-066c-4dc5-886f-a2112411fabc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:06.445497 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1810a1-066c-4dc5-886f-a2112411fabc None admin] RBAC: Adding query filter params (role.id=ca37e43082fa40c6a88b12680a0ca94b, scope.project.id=5633253a5837491c995b7f11753765f8, user.id=2682f5470265422cbaaa91dfd16e8ea6) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:06.445808 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1810a1-066c-4dc5-886f-a2112411fabc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:06.447396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1810a1-066c-4dc5-886f-a2112411fabc None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:06.453866 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 47/91] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:22:06 2022] GET /identity/v3/role_assignments?user.id=2682f5470265422cbaaa91dfd16e8ea6&scope.project.id=5633253a5837491c995b7f11753765f8&role.id=ca37e43082fa40c6a88b12680a0ca94b => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:06.456831 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1810a1-066c-4dc5-886f-a2112411fabc None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:06.458844 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:06.464653 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea6e20-357e-463c-be95-288fb1a5ad13 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:06.530230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8cbdf-94d7-49bf-b9b0-08f0a00fd3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:06.530876 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8cbdf-94d7-49bf-b9b0-08f0a00fd3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:06.531167 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8cbdf-94d7-49bf-b9b0-08f0a00fd3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:06.531447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8cbdf-94d7-49bf-b9b0-08f0a00fd3c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:06.571033 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:06.622294 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1810a1-066c-4dc5-886f-a2112411fabc None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:06.630882 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8cbdf-94d7-49bf-b9b0-08f0a00fd3c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:06.632727 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8cbdf-94d7-49bf-b9b0-08f0a00fd3c6 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:06.668612 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50961080-5ec6-472b-b66c-09c8a73ecda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:06.669306 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50961080-5ec6-472b-b66c-09c8a73ecda7 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:06.670181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50961080-5ec6-472b-b66c-09c8a73ecda7 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:06.670499 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50961080-5ec6-472b-b66c-09c8a73ecda7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:06.749550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50961080-5ec6-472b-b66c-09c8a73ecda7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:06.751589 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50961080-5ec6-472b-b66c-09c8a73ecda7 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:06.757813 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:06.763543 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 45/92] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:06.769350 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8cbdf-94d7-49bf-b9b0-08f0a00fd3c6 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:06.839506 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:06.887496 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 48/93] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:06.893027 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50961080-5ec6-472b-b66c-09c8a73ecda7 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:06.895347 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8cbdf-94d7-49bf-b9b0-08f0a00fd3c6 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:06.932246 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742e36f-3dda-4c8b-a2f6-9aef3e3496a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:06.933336 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742e36f-3dda-4c8b-a2f6-9aef3e3496a3 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:06.933824 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742e36f-3dda-4c8b-a2f6-9aef3e3496a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:06.934301 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742e36f-3dda-4c8b-a2f6-9aef3e3496a3 None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:06.936237 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742e36f-3dda-4c8b-a2f6-9aef3e3496a3 None admin] RBAC: Authorizing `identity:get_project(project_id=5633253a5837491c995b7f11753765f8)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:06.937971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742e36f-3dda-4c8b-a2f6-9aef3e3496a3 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:06.939868 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 46/94] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:06 2022] GET /identity/v3/projects/5633253a5837491c995b7f11753765f8 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:06.945828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742e36f-3dda-4c8b-a2f6-9aef3e3496a3 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:07.029957 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:07.073450 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742e36f-3dda-4c8b-a2f6-9aef3e3496a3 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:07.083736 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50961080-5ec6-472b-b66c-09c8a73ecda7 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:07.118769 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28598b4-597b-495b-84f7-d5569a87cefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:07.119738 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28598b4-597b-495b-84f7-d5569a87cefb None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:07.120489 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28598b4-597b-495b-84f7-d5569a87cefb None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:07.121307 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28598b4-597b-495b-84f7-d5569a87cefb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:07.123651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28598b4-597b-495b-84f7-d5569a87cefb None admin] RBAC: Adding query filter params (role.id=70f3e09bc103410789de8e15c81d9cd3, scope.project.id=5633253a5837491c995b7f11753765f8, user.id=dcba272d6ea54652b4d3c43cf492f3e7) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:07.124200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28598b4-597b-495b-84f7-d5569a87cefb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:07.126356 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28598b4-597b-495b-84f7-d5569a87cefb None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:07.135156 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 47/95] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:22:06 2022] GET /identity/v3/role_assignments?user.id=dcba272d6ea54652b4d3c43cf492f3e7&scope.project.id=5633253a5837491c995b7f11753765f8&role.id=70f3e09bc103410789de8e15c81d9cd3 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:07.168542 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986a91f-94c4-495c-9a5b-38a2e8873fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:07.169378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986a91f-94c4-495c-9a5b-38a2e8873fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:07.170203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986a91f-94c4-495c-9a5b-38a2e8873fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:07.170483 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986a91f-94c4-495c-9a5b-38a2e8873fd4 None admin] PATH_INFO: `/v3/projects/e33f17fdda044f03b50c7ac98a11eed9` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:07.177948 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986a91f-94c4-495c-9a5b-38a2e8873fd4 None admin] RBAC: Authorizing `identity:get_project(project_id=e33f17fdda044f03b50c7ac98a11eed9)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:07.179484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986a91f-94c4-495c-9a5b-38a2e8873fd4 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:07.181302 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 49/96] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:06 2022] GET /identity/v3/projects/e33f17fdda044f03b50c7ac98a11eed9 => generated 312 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:07.187294 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28598b4-597b-495b-84f7-d5569a87cefb None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:07.308690 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28598b4-597b-495b-84f7-d5569a87cefb None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:07.341715 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:07.348286 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74900c2a-77af-422f-935d-98a9be743d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:07.349416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74900c2a-77af-422f-935d-98a9be743d89 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:07.349921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74900c2a-77af-422f-935d-98a9be743d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:07.350466 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74900c2a-77af-422f-935d-98a9be743d89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:07.351460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74900c2a-77af-422f-935d-98a9be743d89 None admin] RBAC: Adding query filter params (role.id=8de48592893c43789e1d40158943fd52, scope.project.id=e33f17fdda044f03b50c7ac98a11eed9, user.id=dcba272d6ea54652b4d3c43cf492f3e7) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:07.351976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74900c2a-77af-422f-935d-98a9be743d89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:07.353712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74900c2a-77af-422f-935d-98a9be743d89 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:07.361160 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 48/97] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:22:07 2022] GET /identity/v3/role_assignments?user.id=dcba272d6ea54652b4d3c43cf492f3e7&scope.project.id=e33f17fdda044f03b50c7ac98a11eed9&role.id=8de48592893c43789e1d40158943fd52 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:09.024429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5443d14-cdda-4059-8065-d43abca4788a None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:09.024967 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5443d14-cdda-4059-8065-d43abca4788a None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:09.025075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5443d14-cdda-4059-8065-d43abca4788a None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:09.026524 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 50/98] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:09.032414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5728f6-17ed-4cf1-a4d1-423665518458 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:09.033233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5728f6-17ed-4cf1-a4d1-423665518458 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:09.033706 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5728f6-17ed-4cf1-a4d1-423665518458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:09.052439 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-5b5728f6-17ed-4cf1-a4d1-423665518458 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:09.053961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-5b5728f6-17ed-4cf1-a4d1-423665518458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:09.149193 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 49/99] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:09.203067 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5443d14-cdda-4059-8065-d43abca4788a None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:09.257671 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:09.348385 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5443d14-cdda-4059-8065-d43abca4788a None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:09.386916 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c623f16-4bee-48fa-a4e2-8f16ac417aac None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:09.387547 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c623f16-4bee-48fa-a4e2-8f16ac417aac None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:09.388028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c623f16-4bee-48fa-a4e2-8f16ac417aac None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:09.390224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 50/100] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:22:09.395355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65e36f-cfb1-4df4-9104-a2b479850e40 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:09.395864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65e36f-cfb1-4df4-9104-a2b479850e40 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:09.396350 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65e36f-cfb1-4df4-9104-a2b479850e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:09.398974 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6902e8f-e0a9-413a-82e7-dc0ed561a77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:09.404914 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6902e8f-e0a9-413a-82e7-dc0ed561a77f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:09.404914 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6902e8f-e0a9-413a-82e7-dc0ed561a77f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:09.404914 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6902e8f-e0a9-413a-82e7-dc0ed561a77f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:09.415651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6902e8f-e0a9-413a-82e7-dc0ed561a77f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:09.421303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-f6902e8f-e0a9-413a-82e7-dc0ed561a77f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 16:22:09.421303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:09.421303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:09.421303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:09.421303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:09.421303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:09.421303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:09.421303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:09.421303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:09.421303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:09.421303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:09.421303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:09.421303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:09.421303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:09.421303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:09.421303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:09.422136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:09.422136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:09.422136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:09.422136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:09.422136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:09.422136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:09.422136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:09.422136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:09.422136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:09.422136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:09.422136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:09.422136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:09.422136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:09.422136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:09.422136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:09.422136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:09.424208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:09.424208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:09.424208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:09.424208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:09.424208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:09.424208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:09.424208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:09.424208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:09.424208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:09.424208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:09.424208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 16:22:09.424208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:09.424208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 51/101] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:09.432522 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6902e8f-e0a9-413a-82e7-dc0ed561a77f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:09.440709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-3c65e36f-cfb1-4df4-9104-a2b479850e40 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:09.442155 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-3c65e36f-cfb1-4df4-9104-a2b479850e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:09.579154 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 51/102] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:09.580934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b02e9-45ea-4dd8-a49b-204be44ff61b None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:09.581434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b02e9-45ea-4dd8-a49b-204be44ff61b None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:09.581988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b02e9-45ea-4dd8-a49b-204be44ff61b None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:09.583437 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 52/103] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:09.589699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda3e8e-0a28-446b-9550-7ded5596274c None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:09.589991 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda3e8e-0a28-446b-9550-7ded5596274c None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:09.590274 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda3e8e-0a28-446b-9550-7ded5596274c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:09.610387 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-5dda3e8e-0a28-446b-9550-7ded5596274c None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:09.611510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-5dda3e8e-0a28-446b-9550-7ded5596274c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:09.626694 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:09.635720 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6902e8f-e0a9-413a-82e7-dc0ed561a77f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:09.671435 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:09.692350 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 53/104] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:09.693474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda3e8e-0a28-446b-9550-7ded5596274c None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:09.728313 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21529bd4-c306-4773-abbc-2af1431c0da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:09.729048 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21529bd4-c306-4773-abbc-2af1431c0da4 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:09.729336 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21529bd4-c306-4773-abbc-2af1431c0da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:09.729618 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21529bd4-c306-4773-abbc-2af1431c0da4 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:09.730352 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21529bd4-c306-4773-abbc-2af1431c0da4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:09.730777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21529bd4-c306-4773-abbc-2af1431c0da4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:09.732821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21529bd4-c306-4773-abbc-2af1431c0da4 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:09.741956 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 52/105] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:09.743624 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64604966-5fe1-4430-890b-1515914fa361 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:09.744017 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64604966-5fe1-4430-890b-1515914fa361 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:09.744400 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64604966-5fe1-4430-890b-1515914fa361 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:09.745795 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 53/106] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:09.748713 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64604966-5fe1-4430-890b-1515914fa361 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:09.844732 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda3e8e-0a28-446b-9550-7ded5596274c None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:09.882993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64604966-5fe1-4430-890b-1515914fa361 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:09.892638 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508f41f-507c-45cf-832c-635c27911db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:09.893343 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508f41f-507c-45cf-832c-635c27911db2 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:09.893640 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508f41f-507c-45cf-832c-635c27911db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:09.893925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508f41f-507c-45cf-832c-635c27911db2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:09.895089 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508f41f-507c-45cf-832c-635c27911db2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:09.897553 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508f41f-507c-45cf-832c-635c27911db2 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:09.899329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 54/107] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:22:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:09.900271 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1273b-543d-4738-b410-c9e5b877fa97 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:09.900559 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1273b-543d-4738-b410-c9e5b877fa97 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:09.900839 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1273b-543d-4738-b410-c9e5b877fa97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:09.918980 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-c9d1273b-543d-4738-b410-c9e5b877fa97 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:09.920226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-c9d1273b-543d-4738-b410-c9e5b877fa97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:09.921936 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b19751-c411-428f-8ecf-46f77324b03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:09.922642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b19751-c411-428f-8ecf-46f77324b03c None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:09.922929 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b19751-c411-428f-8ecf-46f77324b03c None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:09.923211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b19751-c411-428f-8ecf-46f77324b03c None admin] PATH_INFO: `/v3/users/db74378449764a4ea8aa811cf24e5581` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:09.939271 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b19751-c411-428f-8ecf-46f77324b03c None admin] RBAC: Authorizing `identity:get_user(user_id=db74378449764a4ea8aa811cf24e5581)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:09.940756 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b19751-c411-428f-8ecf-46f77324b03c None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:09.943086 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 54/108] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:09 2022] GET /identity/v3/users/db74378449764a4ea8aa811cf24e5581 => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:09.943703 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b19751-c411-428f-8ecf-46f77324b03c None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:09.963289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:09.975871 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:09.997592 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 55/109] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:09.999354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb04c7-471a-41ce-a2f3-9e0ffcc70dc6 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:09.999648 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb04c7-471a-41ce-a2f3-9e0ffcc70dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:09.999930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb04c7-471a-41ce-a2f3-9e0ffcc70dc6 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:10.001364 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 56/110] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:10.001888 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb04c7-471a-41ce-a2f3-9e0ffcc70dc6 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:10.058038 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b19751-c411-428f-8ecf-46f77324b03c None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:10.097077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02541ef8-29c8-4b3a-a7d7-f02f2a986e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:10.098040 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02541ef8-29c8-4b3a-a7d7-f02f2a986e1c None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:10.098331 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02541ef8-29c8-4b3a-a7d7-f02f2a986e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:10.098763 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02541ef8-29c8-4b3a-a7d7-f02f2a986e1c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:10.109052 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02541ef8-29c8-4b3a-a7d7-f02f2a986e1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:10.114008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-02541ef8-29c8-4b3a-a7d7-f02f2a986e1c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:10.114008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:10.114008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:10.114008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:10.114008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:10.114008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:10.114008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:10.114008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:10.114008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:10.114008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:10.114008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:10.114008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:10.114008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:10.114008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:10.114008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:10.114008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:10.114980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:10.114980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:10.114980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:10.114980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:10.114980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:10.114980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:10.114980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:10.114980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:10.114980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:10.114980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:10.114980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:10.114980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:10.114980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:10.114980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:10.114980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:10.114980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:10.120437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:10.120437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:10.120437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:10.120437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:10.120437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:10.120437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:10.120437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:10.120437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:10.120437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:10.120437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:10.120437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:10.120437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:10.120437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 55/111] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:10.120437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02541ef8-29c8-4b3a-a7d7-f02f2a986e1c None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:10.129256 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb04c7-471a-41ce-a2f3-9e0ffcc70dc6 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:10.169061 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852bf96-36ba-432c-a5bf-1323ceec9fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:10.170240 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852bf96-36ba-432c-a5bf-1323ceec9fc3 None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:10.170525 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852bf96-36ba-432c-a5bf-1323ceec9fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:10.170803 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852bf96-36ba-432c-a5bf-1323ceec9fc3 None admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:10.171482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852bf96-36ba-432c-a5bf-1323ceec9fc3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:10.172988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852bf96-36ba-432c-a5bf-1323ceec9fc3 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:10.200299 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 57/112] 10.222.0.74 () {60 vars in 1141 bytes} [Wed Jul 27 16:22:10 2022] POST /identity/v3/projects => generated 302 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:22:10.202079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651378a8-0846-480a-abe3-e6302cc6dbc4 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:10.202375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651378a8-0846-480a-abe3-e6302cc6dbc4 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:10.202649 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651378a8-0846-480a-abe3-e6302cc6dbc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:10.213341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:10.224346 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-651378a8-0846-480a-abe3-e6302cc6dbc4 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:10.225683 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-651378a8-0846-480a-abe3-e6302cc6dbc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:10.254983 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02541ef8-29c8-4b3a-a7d7-f02f2a986e1c None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:10.258756 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:10.308192 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab54199-8c79-4e93-9dbf-cd4762507076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:10.309319 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab54199-8c79-4e93-9dbf-cd4762507076 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:10.309810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab54199-8c79-4e93-9dbf-cd4762507076 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:10.310011 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 58/113] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:10.317154 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651378a8-0846-480a-abe3-e6302cc6dbc4 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:10.318732 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab54199-8c79-4e93-9dbf-cd4762507076 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:10.413895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab54199-8c79-4e93-9dbf-cd4762507076 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:10.415378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab54199-8c79-4e93-9dbf-cd4762507076 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:10.453320 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651378a8-0846-480a-abe3-e6302cc6dbc4 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:10.501035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb06ac-bd5d-439f-bf62-ee209aa6cee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:10.502042 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb06ac-bd5d-439f-bf62-ee209aa6cee6 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:10.502533 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb06ac-bd5d-439f-bf62-ee209aa6cee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:10.503013 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb06ac-bd5d-439f-bf62-ee209aa6cee6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:10.515077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:10.517399 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb06ac-bd5d-439f-bf62-ee209aa6cee6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:10.523778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-96fb06ac-bd5d-439f-bf62-ee209aa6cee6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:22:10.523778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:10.523778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:10.523778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:10.523778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:10.523778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:10.523778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:10.523778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:10.523778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:10.523778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:10.523778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:10.523778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:10.523778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:10.523778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:10.523778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:10.523778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:10.525510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:10.525510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:10.525510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:10.525510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:10.525510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:10.525510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:10.525510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:10.525510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:10.525510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:10.525510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:10.525510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:10.525510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:10.525510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:10.525510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:10.525510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:10.525510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:10.526252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:10.526252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:10.526252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:10.526252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:10.526252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:10.526252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:10.526252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:10.526252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:10.526252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:10.526252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:10.526252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:22:10.526252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:10.526252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 59/114] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:22:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:10.526252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb06ac-bd5d-439f-bf62-ee209aa6cee6 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:10.532339 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 56/115] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:10.533252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab54199-8c79-4e93-9dbf-cd4762507076 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:10.558766 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:10.650613 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab54199-8c79-4e93-9dbf-cd4762507076 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:10.652424 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb06ac-bd5d-439f-bf62-ee209aa6cee6 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:10.688928 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11586318-5990-4cfd-9c07-9fc0c9f17089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:10.689580 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11586318-5990-4cfd-9c07-9fc0c9f17089 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:10.689871 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11586318-5990-4cfd-9c07-9fc0c9f17089 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:10.690207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11586318-5990-4cfd-9c07-9fc0c9f17089 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:10.694725 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09015694-3af1-4798-bd61-57e98b993040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:10.695521 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09015694-3af1-4798-bd61-57e98b993040 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:10.695806 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09015694-3af1-4798-bd61-57e98b993040 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:10.696135 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09015694-3af1-4798-bd61-57e98b993040 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:10.696801 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09015694-3af1-4798-bd61-57e98b993040 None admin] RBAC: Adding query filter params (name=member) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:10.697145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09015694-3af1-4798-bd61-57e98b993040 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:10.698551 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09015694-3af1-4798-bd61-57e98b993040 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:10.701952 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11586318-5990-4cfd-9c07-9fc0c9f17089 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:10.707432 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 60/116] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:10.708068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09015694-3af1-4798-bd61-57e98b993040 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:10.709206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-11586318-5990-4cfd-9c07-9fc0c9f17089 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:10.709206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:10.709206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:10.709206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:10.709206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:10.709206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:10.709206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:10.709206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:10.709206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:10.709206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:10.709206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:10.709206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:10.709206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:10.709206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:10.709206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:10.709206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:10.710072 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:10.710072 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:10.710072 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:10.710072 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:10.710072 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:10.710072 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:10.710072 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:10.710072 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:10.710072 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:10.710072 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:10.710072 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:10.710072 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:10.710072 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:10.710072 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:10.710072 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:10.710072 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:10.710823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:10.710823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:10.710823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:10.710823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:10.710823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:10.710823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:10.710823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:10.710823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:10.710823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:10.710823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:10.710823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:10.710823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:10.714202 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 57/117] 10.222.0.74 () {56 vars in 1114 bytes} [Wed Jul 27 16:22:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:10.715218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11586318-5990-4cfd-9c07-9fc0c9f17089 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:10.756482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:10.799626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:10.817347 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09015694-3af1-4798-bd61-57e98b993040 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:10.827485 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11586318-5990-4cfd-9c07-9fc0c9f17089 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:10.854663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c2153-52f2-4161-be3c-72c795870a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:10.855290 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c2153-52f2-4161-be3c-72c795870a4f None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:10.855577 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c2153-52f2-4161-be3c-72c795870a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:10.855853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c2153-52f2-4161-be3c-72c795870a4f None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:10.856565 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c2153-52f2-4161-be3c-72c795870a4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:10.856873 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c2153-52f2-4161-be3c-72c795870a4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:10.858356 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c2153-52f2-4161-be3c-72c795870a4f None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:10.861824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16a8b6-347a-4b87-8a76-dfe03045d3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:10.862468 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16a8b6-347a-4b87-8a76-dfe03045d3a0 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:10.862752 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16a8b6-347a-4b87-8a76-dfe03045d3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:10.863021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16a8b6-347a-4b87-8a76-dfe03045d3a0 None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:10.864224 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16a8b6-347a-4b87-8a76-dfe03045d3a0 None admin] RBAC: Authorizing `identity:get_project(project_id=5633253a5837491c995b7f11753765f8)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:10.865704 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16a8b6-347a-4b87-8a76-dfe03045d3a0 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:10.866752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 61/118] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:10.872135 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 58/119] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:10 2022] GET /identity/v3/projects/5633253a5837491c995b7f11753765f8 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:10.872135 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16a8b6-347a-4b87-8a76-dfe03045d3a0 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:10.873105 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00013de0-9778-420d-9ed3-0641196a2785 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:10.873394 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00013de0-9778-420d-9ed3-0641196a2785 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:10.873676 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00013de0-9778-420d-9ed3-0641196a2785 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:10.874997 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 62/120] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:10.875672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00013de0-9778-420d-9ed3-0641196a2785 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:10.978135 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16a8b6-347a-4b87-8a76-dfe03045d3a0 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:10.981326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:10.986424 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00013de0-9778-420d-9ed3-0641196a2785 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:11.016551 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f58d8-4704-4060-ade9-e5068e6b04c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:11.017247 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f58d8-4704-4060-ade9-e5068e6b04c2 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:11.017527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f58d8-4704-4060-ade9-e5068e6b04c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:11.017804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f58d8-4704-4060-ade9-e5068e6b04c2 None admin] PATH_INFO: `/v3/users/dcba272d6ea54652b4d3c43cf492f3e7` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:11.018964 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f58d8-4704-4060-ade9-e5068e6b04c2 None admin] RBAC: Authorizing `identity:get_user(user_id=dcba272d6ea54652b4d3c43cf492f3e7)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:11.020409 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f58d8-4704-4060-ade9-e5068e6b04c2 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:11.022030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 59/121] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:10 2022] GET /identity/v3/users/dcba272d6ea54652b4d3c43cf492f3e7 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:11.026495 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e67e24-47e3-4bb9-a26a-41acf949c147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:11.026495 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e67e24-47e3-4bb9-a26a-41acf949c147 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:11.026660 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f58d8-4704-4060-ade9-e5068e6b04c2 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:11.026744 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e67e24-47e3-4bb9-a26a-41acf949c147 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:11.026744 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e67e24-47e3-4bb9-a26a-41acf949c147 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:11.026744 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e67e24-47e3-4bb9-a26a-41acf949c147 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:11.026744 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e67e24-47e3-4bb9-a26a-41acf949c147 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:11.026744 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e67e24-47e3-4bb9-a26a-41acf949c147 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:11.038972 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 63/122] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:22:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:11.038972 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e67e24-47e3-4bb9-a26a-41acf949c147 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:11.051855 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:11.139495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f58d8-4704-4060-ade9-e5068e6b04c2 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:11.147101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e67e24-47e3-4bb9-a26a-41acf949c147 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:11.180530 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b44915-d464-4e5e-a418-420f571b96a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:11.181366 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b44915-d464-4e5e-a418-420f571b96a1 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:11.181963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b44915-d464-4e5e-a418-420f571b96a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:11.182248 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b44915-d464-4e5e-a418-420f571b96a1 None admin] PATH_INFO: `/v3/users/2682f5470265422cbaaa91dfd16e8ea6` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:11.183890 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b44915-d464-4e5e-a418-420f571b96a1 None admin] RBAC: Authorizing `identity:get_user(user_id=2682f5470265422cbaaa91dfd16e8ea6)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:11.186144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b44915-d464-4e5e-a418-420f571b96a1 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:11.193136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 60/123] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:11 2022] GET /identity/v3/users/2682f5470265422cbaaa91dfd16e8ea6 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:11.193281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0e687-6060-42f2-98c1-cc830274b7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:11.193374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca7578-f0cc-4e21-8db2-1f83b1846f35 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:11.193442 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0e687-6060-42f2-98c1-cc830274b7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:11.193528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca7578-f0cc-4e21-8db2-1f83b1846f35 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:11.193597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0e687-6060-42f2-98c1-cc830274b7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:11.193661 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca7578-f0cc-4e21-8db2-1f83b1846f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:11.193724 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0e687-6060-42f2-98c1-cc830274b7f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:11.193724 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0e687-6060-42f2-98c1-cc830274b7f8 None admin] RBAC: Adding query filter params (role.id=1204be9fc6d7483fb28ba04dfa5b3f6e, scope.project.id=5633253a5837491c995b7f11753765f8, user.id=db74378449764a4ea8aa811cf24e5581) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:11.193724 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0e687-6060-42f2-98c1-cc830274b7f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:11.193724 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0e687-6060-42f2-98c1-cc830274b7f8 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:11.198978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 64/124] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:22:11 2022] GET /identity/v3/role_assignments?user.id=db74378449764a4ea8aa811cf24e5581&scope.project.id=5633253a5837491c995b7f11753765f8&role.id=1204be9fc6d7483fb28ba04dfa5b3f6e => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:11.202126 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0e687-6060-42f2-98c1-cc830274b7f8 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:11.218383 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-9fca7578-f0cc-4e21-8db2-1f83b1846f35 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:11.220572 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9fca7578-f0cc-4e21-8db2-1f83b1846f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:11.273454 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:11.291284 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:11.311210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 61/125] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:11.311905 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca7578-f0cc-4e21-8db2-1f83b1846f35 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:11.322802 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0e687-6060-42f2-98c1-cc830274b7f8 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:11.372487 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8984d8b-1fcb-4903-bc1e-248f42f387bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:11.373482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8984d8b-1fcb-4903-bc1e-248f42f387bb None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:11.373910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8984d8b-1fcb-4903-bc1e-248f42f387bb None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:11.374367 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8984d8b-1fcb-4903-bc1e-248f42f387bb None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:11.375871 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8984d8b-1fcb-4903-bc1e-248f42f387bb None admin] RBAC: Authorizing `identity:get_project(project_id=5633253a5837491c995b7f11753765f8)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:11.377699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8984d8b-1fcb-4903-bc1e-248f42f387bb None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:11.379983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 65/126] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:11 2022] GET /identity/v3/projects/5633253a5837491c995b7f11753765f8 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:11.381030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8984d8b-1fcb-4903-bc1e-248f42f387bb None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:11.446226 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca7578-f0cc-4e21-8db2-1f83b1846f35 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:11.494746 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556c91e-8f72-4b10-b4b8-fdc34e697ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:11.495401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556c91e-8f72-4b10-b4b8-fdc34e697ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:11.495687 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556c91e-8f72-4b10-b4b8-fdc34e697ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:11.495974 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556c91e-8f72-4b10-b4b8-fdc34e697ef3 None admin] PATH_INFO: `/v3/users/dcba272d6ea54652b4d3c43cf492f3e7` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:11.496567 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8984d8b-1fcb-4903-bc1e-248f42f387bb None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:11.497184 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556c91e-8f72-4b10-b4b8-fdc34e697ef3 None admin] RBAC: Authorizing `identity:get_user(user_id=dcba272d6ea54652b4d3c43cf492f3e7)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:11.499529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556c91e-8f72-4b10-b4b8-fdc34e697ef3 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:11.502341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 62/127] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:11 2022] GET /identity/v3/users/dcba272d6ea54652b4d3c43cf492f3e7 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:11.502941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556c91e-8f72-4b10-b4b8-fdc34e697ef3 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:11.547685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf953d-6400-4703-b223-af43a427423c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:11.548293 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf953d-6400-4703-b223-af43a427423c None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:11.548588 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf953d-6400-4703-b223-af43a427423c None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:11.549662 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:11.549814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf953d-6400-4703-b223-af43a427423c None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:11.552235 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf953d-6400-4703-b223-af43a427423c None admin] RBAC: Authorizing `identity:get_project(project_id=5633253a5837491c995b7f11753765f8)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:11.556286 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf953d-6400-4703-b223-af43a427423c None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:11.558399 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 66/128] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:11 2022] GET /identity/v3/projects/5633253a5837491c995b7f11753765f8 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:11.559468 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf953d-6400-4703-b223-af43a427423c None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:11.577737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:11.621139 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556c91e-8f72-4b10-b4b8-fdc34e697ef3 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:11.658843 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e9807-ff60-45ed-bef7-432e0f7a7bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:11.659879 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e9807-ff60-45ed-bef7-432e0f7a7bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:11.660368 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e9807-ff60-45ed-bef7-432e0f7a7bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:11.660881 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e9807-ff60-45ed-bef7-432e0f7a7bf2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:11.673361 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e9807-ff60-45ed-bef7-432e0f7a7bf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:11.678758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-889e9807-ff60-45ed-bef7-432e0f7a7bf2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:11.678758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:11.678758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:11.678758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:11.678758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:11.678758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:11.678758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:11.678758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:11.678758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:11.678758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:11.678758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:11.678758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:11.678758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:11.678758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:11.678758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:11.678758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:11.679606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:11.679606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:11.679606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:11.679606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:11.679606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:11.679606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:11.679606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:11.679606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:11.679606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:11.679606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:11.679606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:11.679606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:11.679606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:11.679606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:11.679606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:11.679606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:11.680360 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:11.680360 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:11.680360 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:11.680360 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:11.680360 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:11.680360 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:11.680360 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:11.680360 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:11.680360 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:11.680360 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:11.680360 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:11.680360 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:11.688098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 63/129] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:11.689216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e9807-ff60-45ed-bef7-432e0f7a7bf2 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:11.698051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf953d-6400-4703-b223-af43a427423c None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:11.744976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a382712-9ae7-415e-8ae9-39ea0e80a2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:11.745619 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a382712-9ae7-415e-8ae9-39ea0e80a2eb None admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:11.745906 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a382712-9ae7-415e-8ae9-39ea0e80a2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:11.746190 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a382712-9ae7-415e-8ae9-39ea0e80a2eb None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8/users/dcba272d6ea54652b4d3c43cf492f3e7/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:11.748752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a382712-9ae7-415e-8ae9-39ea0e80a2eb None admin] RBAC: Authorizing `identity:create_grant(project_id=5633253a5837491c995b7f11753765f8, user_id=dcba272d6ea54652b4d3c43cf492f3e7, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:11.750528 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a382712-9ae7-415e-8ae9-39ea0e80a2eb None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:11.784918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 67/130] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:22:11 2022] PUT /identity/v3/projects/5633253a5837491c995b7f11753765f8/users/dcba272d6ea54652b4d3c43cf492f3e7/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:22:11.784918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a382712-9ae7-415e-8ae9-39ea0e80a2eb None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:11.843422 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:11.851940 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e9807-ff60-45ed-bef7-432e0f7a7bf2 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:11.869823 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:11.910785 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a382712-9ae7-415e-8ae9-39ea0e80a2eb None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:11.954581 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e3ce1-d1f9-4c54-8b12-1045b03a8791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:11.955820 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e3ce1-d1f9-4c54-8b12-1045b03a8791 None admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:11.956311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e3ce1-d1f9-4c54-8b12-1045b03a8791 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:11.956823 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e3ce1-d1f9-4c54-8b12-1045b03a8791 None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8/users/2682f5470265422cbaaa91dfd16e8ea6/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:11.957640 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d06e9-bf0e-4cbc-8dcf-dff750350557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:11.958437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d06e9-bf0e-4cbc-8dcf-dff750350557 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:11.958788 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d06e9-bf0e-4cbc-8dcf-dff750350557 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:11.959172 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d06e9-bf0e-4cbc-8dcf-dff750350557 None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:11.959597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e3ce1-d1f9-4c54-8b12-1045b03a8791 None admin] RBAC: Authorizing `identity:create_grant(project_id=5633253a5837491c995b7f11753765f8, user_id=2682f5470265422cbaaa91dfd16e8ea6, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:11.961357 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e3ce1-d1f9-4c54-8b12-1045b03a8791 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:11.966609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d06e9-bf0e-4cbc-8dcf-dff750350557 None admin] RBAC: Authorizing `identity:get_project(project_id=5633253a5837491c995b7f11753765f8)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:11.969336 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d06e9-bf0e-4cbc-8dcf-dff750350557 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:11.971764 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 64/131] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:11 2022] GET /identity/v3/projects/5633253a5837491c995b7f11753765f8 => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:11.976867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d06e9-bf0e-4cbc-8dcf-dff750350557 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:11.983434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 68/132] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:22:11 2022] PUT /identity/v3/projects/5633253a5837491c995b7f11753765f8/users/2682f5470265422cbaaa91dfd16e8ea6/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:22:11.984576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e3ce1-d1f9-4c54-8b12-1045b03a8791 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:12.138979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d06e9-bf0e-4cbc-8dcf-dff750350557 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:12.144113 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e3ce1-d1f9-4c54-8b12-1045b03a8791 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:12.176989 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:12.187305 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f8576-c573-4b0f-a149-d42b57305d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:12.188091 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f8576-c573-4b0f-a149-d42b57305d7e None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:12.188480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f8576-c573-4b0f-a149-d42b57305d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:12.188864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f8576-c573-4b0f-a149-d42b57305d7e None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:12.189741 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f8576-c573-4b0f-a149-d42b57305d7e None admin] RBAC: Adding query filter params (name=member) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:12.190157 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f8576-c573-4b0f-a149-d42b57305d7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:12.191669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f8576-c573-4b0f-a149-d42b57305d7e None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:12.192537 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d3ec9-d3ec-4b06-8995-6acc54cb7728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:12.193435 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d3ec9-d3ec-4b06-8995-6acc54cb7728 None admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:12.193859 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d3ec9-d3ec-4b06-8995-6acc54cb7728 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:12.194286 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d3ec9-d3ec-4b06-8995-6acc54cb7728 None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8/users/dcba272d6ea54652b4d3c43cf492f3e7/roles/70f3e09bc103410789de8e15c81d9cd3` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:12.202064 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 65/133] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:12.202666 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d3ec9-d3ec-4b06-8995-6acc54cb7728 None admin] RBAC: Authorizing `identity:create_grant(project_id=5633253a5837491c995b7f11753765f8, user_id=dcba272d6ea54652b4d3c43cf492f3e7, role_id=70f3e09bc103410789de8e15c81d9cd3)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:12.204347 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d3ec9-d3ec-4b06-8995-6acc54cb7728 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:12.210189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f8576-c573-4b0f-a149-d42b57305d7e None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:12.230041 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 69/134] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:22:11 2022] PUT /identity/v3/projects/5633253a5837491c995b7f11753765f8/users/dcba272d6ea54652b4d3c43cf492f3e7/roles/70f3e09bc103410789de8e15c81d9cd3 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:22:12.264923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:12.376626 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f8576-c573-4b0f-a149-d42b57305d7e None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:12.406145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ab120-3bbf-4b99-ad96-730bd64c1b48 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:12.406502 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ab120-3bbf-4b99-ad96-730bd64c1b48 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:12.407352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ab120-3bbf-4b99-ad96-730bd64c1b48 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:12.408682 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 70/135] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:12.415613 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61b61d-75eb-4811-9751-b11979f0935a None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:12.415898 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61b61d-75eb-4811-9751-b11979f0935a None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:12.416178 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61b61d-75eb-4811-9751-b11979f0935a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:12.428995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e372974-39b9-4e6a-a553-66f4a273d0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:12.429656 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e372974-39b9-4e6a-a553-66f4a273d0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:12.429937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e372974-39b9-4e6a-a553-66f4a273d0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:12.430214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e372974-39b9-4e6a-a553-66f4a273d0d8 None admin] PATH_INFO: `/v3/users/dcba272d6ea54652b4d3c43cf492f3e7` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:12.434193 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e372974-39b9-4e6a-a553-66f4a273d0d8 None admin] RBAC: Authorizing `identity:get_user(user_id=dcba272d6ea54652b4d3c43cf492f3e7)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:12.436303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e372974-39b9-4e6a-a553-66f4a273d0d8 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:12.438219 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 66/136] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:12 2022] GET /identity/v3/users/dcba272d6ea54652b4d3c43cf492f3e7 => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:12.440169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-6a61b61d-75eb-4811-9751-b11979f0935a None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:12.441620 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-6a61b61d-75eb-4811-9751-b11979f0935a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:12.443715 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e372974-39b9-4e6a-a553-66f4a273d0d8 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:12.529954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 71/137] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:12.554428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:12.560941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e372974-39b9-4e6a-a553-66f4a273d0d8 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:12.563266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61b61d-75eb-4811-9751-b11979f0935a None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:12.618552 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1101838d-276d-42f3-8a97-7da249de16a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:12.619595 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101838d-276d-42f3-8a97-7da249de16a8 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:12.619888 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101838d-276d-42f3-8a97-7da249de16a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:12.620170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101838d-276d-42f3-8a97-7da249de16a8 None admin] PATH_INFO: `/v3/projects/e33f17fdda044f03b50c7ac98a11eed9` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:12.621674 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1101838d-276d-42f3-8a97-7da249de16a8 None admin] RBAC: Authorizing `identity:get_project(project_id=e33f17fdda044f03b50c7ac98a11eed9)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:12.623229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1101838d-276d-42f3-8a97-7da249de16a8 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:12.625431 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 67/138] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:12 2022] GET /identity/v3/projects/e33f17fdda044f03b50c7ac98a11eed9 => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:12.631545 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1101838d-276d-42f3-8a97-7da249de16a8 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:12.751012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1101838d-276d-42f3-8a97-7da249de16a8 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:12.788859 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4dadc-a8e7-4e3e-81ca-a3fd99cf61c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:12.789610 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4dadc-a8e7-4e3e-81ca-a3fd99cf61c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:12.790041 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4dadc-a8e7-4e3e-81ca-a3fd99cf61c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:12.790452 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4dadc-a8e7-4e3e-81ca-a3fd99cf61c0 None admin] PATH_INFO: `/v3/projects/e33f17fdda044f03b50c7ac98a11eed9/users/dcba272d6ea54652b4d3c43cf492f3e7/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:12.792416 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4dadc-a8e7-4e3e-81ca-a3fd99cf61c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e33f17fdda044f03b50c7ac98a11eed9, user_id=dcba272d6ea54652b4d3c43cf492f3e7, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:12.792645 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61b61d-75eb-4811-9751-b11979f0935a None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:12.793978 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4dadc-a8e7-4e3e-81ca-a3fd99cf61c0 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:12.811041 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 68/139] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:22:12 2022] PUT /identity/v3/projects/e33f17fdda044f03b50c7ac98a11eed9/users/dcba272d6ea54652b4d3c43cf492f3e7/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:22:12.840169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:12.870191 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e839e78-894d-4061-9dfe-a762bc9d9937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:12.870925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e839e78-894d-4061-9dfe-a762bc9d9937 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:12.871195 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e839e78-894d-4061-9dfe-a762bc9d9937 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:12.871499 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e839e78-894d-4061-9dfe-a762bc9d9937 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:12.877686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e839e78-894d-4061-9dfe-a762bc9d9937 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:12.879202 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e839e78-894d-4061-9dfe-a762bc9d9937 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:12.880941 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 72/140] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:22:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:12.886750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4dadc-a8e7-4e3e-81ca-a3fd99cf61c0 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:12.926213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:13.023186 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4dadc-a8e7-4e3e-81ca-a3fd99cf61c0 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:13.059265 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4da108-1cd9-4563-a946-5ae88b753380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:13.060523 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4da108-1cd9-4563-a946-5ae88b753380 None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:13.060907 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4da108-1cd9-4563-a946-5ae88b753380 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:13.061271 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4da108-1cd9-4563-a946-5ae88b753380 None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:13.062010 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4da108-1cd9-4563-a946-5ae88b753380 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:13.063557 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4da108-1cd9-4563-a946-5ae88b753380 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:13.095676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 69/141] 10.222.0.74 () {60 vars in 1132 bytes} [Wed Jul 27 16:22:12 2022] POST /identity/v3/users => generated 283 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:22:13.832047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824bd66-7dd8-45bf-ba9d-cae5829ccd65 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:13.832546 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824bd66-7dd8-45bf-ba9d-cae5829ccd65 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:13.832619 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824bd66-7dd8-45bf-ba9d-cae5829ccd65 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:13.834123 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 73/142] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:13.839500 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7350bbb-cb74-4e48-891a-bd62115c7003 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:13.840294 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7350bbb-cb74-4e48-891a-bd62115c7003 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:13.840787 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7350bbb-cb74-4e48-891a-bd62115c7003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:13.864663 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-b7350bbb-cb74-4e48-891a-bd62115c7003 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:13.865908 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b7350bbb-cb74-4e48-891a-bd62115c7003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:13.946685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 70/143] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:13.971048 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824bd66-7dd8-45bf-ba9d-cae5829ccd65 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:14.102183 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824bd66-7dd8-45bf-ba9d-cae5829ccd65 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:14.140838 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148ed12-cda1-4775-b980-d0c44515165e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:14.141521 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148ed12-cda1-4775-b980-d0c44515165e None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:14.141806 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148ed12-cda1-4775-b980-d0c44515165e None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:14.142083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148ed12-cda1-4775-b980-d0c44515165e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:14.155343 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3148ed12-cda1-4775-b980-d0c44515165e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:14.160426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-3148ed12-cda1-4775-b980-d0c44515165e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 16:22:14.160426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:14.160426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:14.160426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:14.160426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:14.160426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:14.160426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:14.160426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:14.160426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:14.160426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:14.160426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:14.160426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:14.160426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:14.160426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:14.160426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:14.160426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:14.161297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:14.161297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:14.161297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:14.161297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:14.161297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:14.161297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:14.161297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:14.161297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:14.161297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:14.161297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:14.161297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:14.161297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:14.161297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:14.161297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:14.161297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:14.161297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:14.162058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:14.162058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:14.162058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:14.162058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:14.162058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:14.162058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:14.162058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:14.162058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:14.162058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:14.162058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:14.162058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 16:22:14.162058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:14.164144 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 74/144] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:14.172932 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7350bbb-cb74-4e48-891a-bd62115c7003 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:14.192653 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:14.317513 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7350bbb-cb74-4e48-891a-bd62115c7003 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:14.365202 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7ddfd-d4ac-41fd-add1-a50df584612b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:14.365748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7ddfd-d4ac-41fd-add1-a50df584612b None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:14.366031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7ddfd-d4ac-41fd-add1-a50df584612b None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:14.366301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7ddfd-d4ac-41fd-add1-a50df584612b None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:14.367029 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d7ddfd-d4ac-41fd-add1-a50df584612b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:14.367338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d7ddfd-d4ac-41fd-add1-a50df584612b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:14.368762 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d7ddfd-d4ac-41fd-add1-a50df584612b None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:14.379628 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 71/145] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:14.385384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148ed12-cda1-4775-b980-d0c44515165e None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:14.437242 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:14.490005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148ed12-cda1-4775-b980-d0c44515165e None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:14.525131 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a0782-469e-4daa-9b62-f012d79df63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:14.525777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a0782-469e-4daa-9b62-f012d79df63d None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:14.526056 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a0782-469e-4daa-9b62-f012d79df63d None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:14.526328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a0782-469e-4daa-9b62-f012d79df63d None admin] PATH_INFO: `/v3/users/db74378449764a4ea8aa811cf24e5581` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:14.527487 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a0782-469e-4daa-9b62-f012d79df63d None admin] RBAC: Authorizing `identity:get_user(user_id=db74378449764a4ea8aa811cf24e5581)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:14.529489 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a0782-469e-4daa-9b62-f012d79df63d None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:14.531267 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 75/146] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:14 2022] GET /identity/v3/users/db74378449764a4ea8aa811cf24e5581 => generated 289 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:14.536708 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7ddfd-d4ac-41fd-add1-a50df584612b None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:14.610824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:14.643226 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7ddfd-d4ac-41fd-add1-a50df584612b None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:14.678294 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e7fb4-0f7d-471b-899e-b4719487d0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:14.678951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e7fb4-0f7d-471b-899e-b4719487d0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:14.679229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e7fb4-0f7d-471b-899e-b4719487d0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:14.679502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e7fb4-0f7d-471b-899e-b4719487d0e7 None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:14.680669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e7fb4-0f7d-471b-899e-b4719487d0e7 None admin] RBAC: Authorizing `identity:get_project(project_id=5633253a5837491c995b7f11753765f8)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:14.682176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e7fb4-0f7d-471b-899e-b4719487d0e7 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:14.684198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 72/147] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:14 2022] GET /identity/v3/projects/5633253a5837491c995b7f11753765f8 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:14.690513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a0782-469e-4daa-9b62-f012d79df63d None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:14.797400 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a0782-469e-4daa-9b62-f012d79df63d None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:14.816453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:14.836111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a9dda-d414-4370-aa1f-c4a126a72f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:14.836853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a9dda-d414-4370-aa1f-c4a126a72f68 None admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:14.837105 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a9dda-d414-4370-aa1f-c4a126a72f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:14.837448 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a9dda-d414-4370-aa1f-c4a126a72f68 None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8/users/db74378449764a4ea8aa811cf24e5581/roles/1204be9fc6d7483fb28ba04dfa5b3f6e` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:14.839663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a9dda-d414-4370-aa1f-c4a126a72f68 None admin] RBAC: Authorizing `identity:create_grant(project_id=5633253a5837491c995b7f11753765f8, user_id=db74378449764a4ea8aa811cf24e5581, role_id=1204be9fc6d7483fb28ba04dfa5b3f6e)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:14.841200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a9dda-d414-4370-aa1f-c4a126a72f68 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:14.856401 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 76/148] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:22:14 2022] PUT /identity/v3/projects/5633253a5837491c995b7f11753765f8/users/db74378449764a4ea8aa811cf24e5581/roles/1204be9fc6d7483fb28ba04dfa5b3f6e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:22:15.069782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a961f-ff0a-40c8-8fa5-ceba3198ddf7 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:15.070282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a961f-ff0a-40c8-8fa5-ceba3198ddf7 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:15.070382 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a961f-ff0a-40c8-8fa5-ceba3198ddf7 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:15.071710 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 73/149] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:22:15.080019 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e08205-8c25-40da-9773-20bce9faa1c7 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:15.080311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e08205-8c25-40da-9773-20bce9faa1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:15.080588 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e08205-8c25-40da-9773-20bce9faa1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:15.100646 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-70e08205-8c25-40da-9773-20bce9faa1c7 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:15.101859 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-70e08205-8c25-40da-9773-20bce9faa1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:15.188080 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 77/150] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:15.214098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1a16a-d4d9-4d50-9599-31e21316c112 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:15.214381 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1a16a-d4d9-4d50-9599-31e21316c112 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:15.214907 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd1a16a-d4d9-4d50-9599-31e21316c112 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:15.216523 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 74/151] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:15.222052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c631cb5-4a57-4c3e-a590-5c4f0e210bc5 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:15.222342 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c631cb5-4a57-4c3e-a590-5c4f0e210bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:15.222619 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c631cb5-4a57-4c3e-a590-5c4f0e210bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:15.229925 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1a16a-d4d9-4d50-9599-31e21316c112 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:15.251271 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-9c631cb5-4a57-4c3e-a590-5c4f0e210bc5 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:15.252471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-9c631cb5-4a57-4c3e-a590-5c4f0e210bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:15.356698 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 78/152] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:15.397202 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd1a16a-d4d9-4d50-9599-31e21316c112 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:15.426526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c631cb5-4a57-4c3e-a590-5c4f0e210bc5 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:15.436924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b788cd-1744-4c6c-9f31-68d3b7885f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:15.436924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b788cd-1744-4c6c-9f31-68d3b7885f32 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:15.436924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b788cd-1744-4c6c-9f31-68d3b7885f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:15.436924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b788cd-1744-4c6c-9f31-68d3b7885f32 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:15.448912 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b788cd-1744-4c6c-9f31-68d3b7885f32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:15.452667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-f0b788cd-1744-4c6c-9f31-68d3b7885f32 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:15.452667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:15.452667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:15.452667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:15.452667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:15.452667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:15.452667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:15.452667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:15.452667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:15.452667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:15.452667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:15.452667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:15.452667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:15.452667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:15.452667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:15.452667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:15.453750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:15.453750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:15.453750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:15.453750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:15.453750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:15.453750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:15.453750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:15.453750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:15.453750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:15.453750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:15.453750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:15.453750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:15.453750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:15.453750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:15.453750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:15.453750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:15.454592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:15.454592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:15.454592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:15.454592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:15.454592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:15.454592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:15.454592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:15.454592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:15.454592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:15.454592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:15.454592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:15.454592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:15.457073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:15.464405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 75/153] 10.222.0.74 () {56 vars in 1114 bytes} [Wed Jul 27 16:22:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:15.465783 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b788cd-1744-4c6c-9f31-68d3b7885f32 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:15.504156 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:15.570166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b788cd-1744-4c6c-9f31-68d3b7885f32 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:15.607459 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c631cb5-4a57-4c3e-a590-5c4f0e210bc5 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:15.607829 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6f2d2-f10c-4b04-9ee2-668e49b3a5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:15.608671 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6f2d2-f10c-4b04-9ee2-668e49b3a5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:15.609228 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6f2d2-f10c-4b04-9ee2-668e49b3a5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:15.609705 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6f2d2-f10c-4b04-9ee2-668e49b3a5b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:15.610681 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6f2d2-f10c-4b04-9ee2-668e49b3a5b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:15.611230 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6f2d2-f10c-4b04-9ee2-668e49b3a5b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:15.612870 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6f2d2-f10c-4b04-9ee2-668e49b3a5b8 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:15.621828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 76/154] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:22:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:15.631099 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6f2d2-f10c-4b04-9ee2-668e49b3a5b8 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:15.658785 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac387a3f-3efb-4062-89d0-3a2b72e8e4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:15.659446 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac387a3f-3efb-4062-89d0-3a2b72e8e4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:15.659726 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac387a3f-3efb-4062-89d0-3a2b72e8e4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:15.659992 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac387a3f-3efb-4062-89d0-3a2b72e8e4f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:15.689659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac387a3f-3efb-4062-89d0-3a2b72e8e4f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:15.694658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-ac387a3f-3efb-4062-89d0-3a2b72e8e4f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:15.694658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:15.694658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:15.694658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:15.694658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:15.694658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:15.694658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:15.694658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:15.694658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:15.694658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:15.694658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:15.694658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:15.694658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:15.694658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:15.694658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:15.694658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:15.695526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:15.695526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:15.695526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:15.695526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:15.695526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:15.695526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:15.695526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:15.695526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:15.695526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:15.695526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:15.695526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:15.695526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:15.695526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:15.695526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:15.695526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:15.695526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:15.696488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:15.696488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:15.696488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:15.696488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:15.696488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:15.696488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:15.696488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:15.696488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:15.696488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:15.696488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:15.696488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:15.696488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:15.696488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 79/155] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:22:15.711935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac387a3f-3efb-4062-89d0-3a2b72e8e4f7 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:15.778378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:15.878773 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6f2d2-f10c-4b04-9ee2-668e49b3a5b8 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:15.925939 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:15.928400 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4621998-865a-4a92-850f-258953593c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:15.929321 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4621998-865a-4a92-850f-258953593c18 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:15.929895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4621998-865a-4a92-850f-258953593c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:15.930495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4621998-865a-4a92-850f-258953593c18 None admin] PATH_INFO: `/v3/users/dcba272d6ea54652b4d3c43cf492f3e7` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:15.931997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4621998-865a-4a92-850f-258953593c18 None admin] RBAC: Authorizing `identity:get_user(user_id=dcba272d6ea54652b4d3c43cf492f3e7)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:15.933920 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4621998-865a-4a92-850f-258953593c18 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:15.940952 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac387a3f-3efb-4062-89d0-3a2b72e8e4f7 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:15.942088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 77/156] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:15 2022] GET /identity/v3/users/dcba272d6ea54652b4d3c43cf492f3e7 => generated 275 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:15.957720 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4621998-865a-4a92-850f-258953593c18 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:15.999214 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7a756-d535-4f2e-8495-4233f06bbf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:15.999985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7a756-d535-4f2e-8495-4233f06bbf70 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:16.000267 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7a756-d535-4f2e-8495-4233f06bbf70 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:16.000536 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7a756-d535-4f2e-8495-4233f06bbf70 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:16.001280 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7a756-d535-4f2e-8495-4233f06bbf70 None admin] RBAC: Adding query filter params (name=member) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:16.001579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7a756-d535-4f2e-8495-4233f06bbf70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:16.003238 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7a756-d535-4f2e-8495-4233f06bbf70 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:16.017999 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 80/157] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:16.028380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7a756-d535-4f2e-8495-4233f06bbf70 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:16.123863 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4621998-865a-4a92-850f-258953593c18 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:16.153698 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7a756-d535-4f2e-8495-4233f06bbf70 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:16.175999 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:16.185333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b727f-3b70-4b03-8ef7-0d09acf3e84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:16.186240 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b727f-3b70-4b03-8ef7-0d09acf3e84a None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:16.186629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b727f-3b70-4b03-8ef7-0d09acf3e84a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:16.187138 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b727f-3b70-4b03-8ef7-0d09acf3e84a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:16.201687 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508092e7-ee9a-402f-a407-bfd06d969872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:16.202346 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508092e7-ee9a-402f-a407-bfd06d969872 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:16.202626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508092e7-ee9a-402f-a407-bfd06d969872 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:16.202901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508092e7-ee9a-402f-a407-bfd06d969872 None admin] PATH_INFO: `/v3/users/dcba272d6ea54652b4d3c43cf492f3e7` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:16.206158 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508092e7-ee9a-402f-a407-bfd06d969872 None admin] RBAC: Authorizing `identity:get_user(user_id=dcba272d6ea54652b4d3c43cf492f3e7)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:16.207645 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508092e7-ee9a-402f-a407-bfd06d969872 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:16.209316 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 81/158] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:16 2022] GET /identity/v3/users/dcba272d6ea54652b4d3c43cf492f3e7 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:16.210472 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326ca0a-3f86-4578-ad17-34ecb759f799 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:16.210754 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326ca0a-3f86-4578-ad17-34ecb759f799 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:16.211019 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326ca0a-3f86-4578-ad17-34ecb759f799 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:16.212396 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 82/159] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:16.222638 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583f7e9-b988-4522-9207-f73fabd3d717 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:16.222923 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583f7e9-b988-4522-9207-f73fabd3d717 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:16.223187 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583f7e9-b988-4522-9207-f73fabd3d717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:16.257937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-b583f7e9-b988-4522-9207-f73fabd3d717 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:16.259147 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b583f7e9-b988-4522-9207-f73fabd3d717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:16.282136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:16.293608 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b727f-3b70-4b03-8ef7-0d09acf3e84a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:16.295330 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b727f-3b70-4b03-8ef7-0d09acf3e84a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:16.367534 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 83/160] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:16.368405 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583f7e9-b988-4522-9207-f73fabd3d717 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:16.463321 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 78/161] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 16:22:16.464954 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b727f-3b70-4b03-8ef7-0d09acf3e84a None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:16.524680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:16.556995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583f7e9-b988-4522-9207-f73fabd3d717 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:16.615844 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e2091-8a42-4aa7-bb66-bc133c299d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:16.616482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e2091-8a42-4aa7-bb66-bc133c299d8f None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:16.616768 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e2091-8a42-4aa7-bb66-bc133c299d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:16.617071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e2091-8a42-4aa7-bb66-bc133c299d8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:16.657843 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:16.669137 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b727f-3b70-4b03-8ef7-0d09acf3e84a None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:16.721680 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e2091-8a42-4aa7-bb66-bc133c299d8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:16.723479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e2091-8a42-4aa7-bb66-bc133c299d8f None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:16.793615 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a05ab-4234-4f25-b4ad-182bec4b4641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:16.794311 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a05ab-4234-4f25-b4ad-182bec4b4641 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:16.794601 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a05ab-4234-4f25-b4ad-182bec4b4641 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:16.794886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a05ab-4234-4f25-b4ad-182bec4b4641 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:16.809171 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a05ab-4234-4f25-b4ad-182bec4b4641 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:16.814770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-516a05ab-4234-4f25-b4ad-182bec4b4641 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:22:16.814770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:16.814770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:16.814770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:16.814770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:16.814770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:16.814770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:16.814770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:16.814770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:16.814770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:16.814770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:16.814770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:16.814770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:16.814770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:16.814770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:16.814770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:16.815609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:16.815609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:16.815609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:16.815609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:16.815609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:16.815609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:16.815609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:16.815609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:16.815609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:16.815609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:16.815609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:16.815609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:16.815609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:16.815609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:16.815609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:16.815609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:16.816840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:16.816840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:16.816840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:16.816840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:16.816840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:16.816840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:16.816840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:16.816840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:16.816840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:16.816840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:16.816840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:22:16.816840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:16.816840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 79/162] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:22:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:16.816840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a05ab-4234-4f25-b4ad-182bec4b4641 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:16.913633 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:16.924302 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a05ab-4234-4f25-b4ad-182bec4b4641 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:16.926472 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 84/163] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:16.927664 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a5057-fd4b-48a8-b9a9-eb93e255cb74 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:16.927953 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a5057-fd4b-48a8-b9a9-eb93e255cb74 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:16.928228 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a5057-fd4b-48a8-b9a9-eb93e255cb74 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:16.929732 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 85/164] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:16.930442 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a5057-fd4b-48a8-b9a9-eb93e255cb74 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:16.962021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423c7df-70e8-497e-97ea-89227dcf128f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:16.962682 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423c7df-70e8-497e-97ea-89227dcf128f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:16.962970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423c7df-70e8-497e-97ea-89227dcf128f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:16.963248 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423c7df-70e8-497e-97ea-89227dcf128f None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:16.964426 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5423c7df-70e8-497e-97ea-89227dcf128f None admin] RBAC: Authorizing `identity:get_project(project_id=5633253a5837491c995b7f11753765f8)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:16.965929 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5423c7df-70e8-497e-97ea-89227dcf128f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:16.968173 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 80/165] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:16 2022] GET /identity/v3/projects/5633253a5837491c995b7f11753765f8 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:16.969114 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423c7df-70e8-497e-97ea-89227dcf128f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:17.100277 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423c7df-70e8-497e-97ea-89227dcf128f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:17.172583 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:17.210926 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dba3f6-9344-45af-b2a3-82014723b5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:17.211779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dba3f6-9344-45af-b2a3-82014723b5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:17.212091 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dba3f6-9344-45af-b2a3-82014723b5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:17.212398 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dba3f6-9344-45af-b2a3-82014723b5d1 None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:17.221424 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dba3f6-9344-45af-b2a3-82014723b5d1 None admin] RBAC: Authorizing `identity:get_project(project_id=5633253a5837491c995b7f11753765f8)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:17.223102 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dba3f6-9344-45af-b2a3-82014723b5d1 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:17.225392 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 81/166] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:16 2022] GET /identity/v3/projects/5633253a5837491c995b7f11753765f8 => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:17.226723 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8784b-4731-4999-af2f-7c8ccb33504a None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:17.227157 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8784b-4731-4999-af2f-7c8ccb33504a None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:17.227593 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8784b-4731-4999-af2f-7c8ccb33504a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:17.262542 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-ece8784b-4731-4999-af2f-7c8ccb33504a None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:17.264078 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ece8784b-4731-4999-af2f-7c8ccb33504a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:17.269618 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a5057-fd4b-48a8-b9a9-eb93e255cb74 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:17.309275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:17.358675 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6afab-9fcd-4451-ba3e-24113f26a178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:17.359230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6afab-9fcd-4451-ba3e-24113f26a178 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:17.359513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6afab-9fcd-4451-ba3e-24113f26a178 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:17.359791 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6afab-9fcd-4451-ba3e-24113f26a178 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:17.360516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6afab-9fcd-4451-ba3e-24113f26a178 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:17.360823 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6afab-9fcd-4451-ba3e-24113f26a178 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:17.362235 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6afab-9fcd-4451-ba3e-24113f26a178 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:17.380442 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 86/167] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:17.381325 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6afab-9fcd-4451-ba3e-24113f26a178 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:17.447558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 82/168] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:17.448446 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8784b-4731-4999-af2f-7c8ccb33504a None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:17.612747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6afab-9fcd-4451-ba3e-24113f26a178 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:17.644916 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:17.664403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93faf0f7-7833-4c76-b946-5781125559ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:17.665311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93faf0f7-7833-4c76-b946-5781125559ac None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:17.665738 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93faf0f7-7833-4c76-b946-5781125559ac None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:17.666172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93faf0f7-7833-4c76-b946-5781125559ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:17.667102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93faf0f7-7833-4c76-b946-5781125559ac None admin] RBAC: Adding query filter params (role.id=70f3e09bc103410789de8e15c81d9cd3, scope.project.id=5633253a5837491c995b7f11753765f8, user.id=dcba272d6ea54652b4d3c43cf492f3e7) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:17.667653 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93faf0f7-7833-4c76-b946-5781125559ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:17.669805 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93faf0f7-7833-4c76-b946-5781125559ac None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:17.676210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 87/169] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:22:17 2022] GET /identity/v3/role_assignments?user.id=dcba272d6ea54652b4d3c43cf492f3e7&scope.project.id=5633253a5837491c995b7f11753765f8&role.id=70f3e09bc103410789de8e15c81d9cd3 => generated 608 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:17.677310 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93faf0f7-7833-4c76-b946-5781125559ac None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:17.683875 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8784b-4731-4999-af2f-7c8ccb33504a None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:17.761687 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179df4a-94c7-40b1-8297-be21054feb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:17.762325 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179df4a-94c7-40b1-8297-be21054feb77 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:17.762768 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179df4a-94c7-40b1-8297-be21054feb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:17.763722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179df4a-94c7-40b1-8297-be21054feb77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:17.764692 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179df4a-94c7-40b1-8297-be21054feb77 None admin] RBAC: Adding query filter params (role.id=8de48592893c43789e1d40158943fd52, scope.project.id=5633253a5837491c995b7f11753765f8, user.id=dcba272d6ea54652b4d3c43cf492f3e7) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:17.765245 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179df4a-94c7-40b1-8297-be21054feb77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:17.766866 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179df4a-94c7-40b1-8297-be21054feb77 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:17.773730 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 83/170] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:22:17 2022] GET /identity/v3/role_assignments?user.id=dcba272d6ea54652b4d3c43cf492f3e7&scope.project.id=5633253a5837491c995b7f11753765f8&role.id=8de48592893c43789e1d40158943fd52 => generated 608 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:17.774697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179df4a-94c7-40b1-8297-be21054feb77 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:17.788355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93faf0f7-7833-4c76-b946-5781125559ac None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:17.820746 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:17.832031 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d52490-80ea-432d-a0bd-d4083ab98092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:17.833250 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d52490-80ea-432d-a0bd-d4083ab98092 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:17.833757 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d52490-80ea-432d-a0bd-d4083ab98092 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:17.834252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d52490-80ea-432d-a0bd-d4083ab98092 None admin] PATH_INFO: `/v3/users/2682f5470265422cbaaa91dfd16e8ea6` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:17.835951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d52490-80ea-432d-a0bd-d4083ab98092 None admin] RBAC: Authorizing `identity:get_user(user_id=2682f5470265422cbaaa91dfd16e8ea6)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:17.837749 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d52490-80ea-432d-a0bd-d4083ab98092 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:17.839890 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 88/171] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:17 2022] GET /identity/v3/users/2682f5470265422cbaaa91dfd16e8ea6 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:17.841579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ce0dd-cd9a-464b-a17f-1bcfdb97545f None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:17.842121 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ce0dd-cd9a-464b-a17f-1bcfdb97545f None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:17.842653 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ce0dd-cd9a-464b-a17f-1bcfdb97545f None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:17.844303 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 89/172] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:17.845565 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ce0dd-cd9a-464b-a17f-1bcfdb97545f None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:17.909385 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:17.947687 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179df4a-94c7-40b1-8297-be21054feb77 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:17.982310 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ce0dd-cd9a-464b-a17f-1bcfdb97545f None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:17.998695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755982e-0acc-4071-94c2-cb270831103f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:17.999419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755982e-0acc-4071-94c2-cb270831103f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:17.999703 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755982e-0acc-4071-94c2-cb270831103f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:17.999976 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755982e-0acc-4071-94c2-cb270831103f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:18.014923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d755982e-0acc-4071-94c2-cb270831103f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:18.024036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763d104-468b-44ba-98d2-b4e856a23991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:18.025141 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763d104-468b-44ba-98d2-b4e856a23991 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:18.025670 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763d104-468b-44ba-98d2-b4e856a23991 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:18.026243 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763d104-468b-44ba-98d2-b4e856a23991 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:18.029151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-d755982e-0acc-4071-94c2-cb270831103f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:18.029151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:18.029151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:18.029151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:18.029151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:18.029151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:18.029151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:18.029151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:18.029151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:18.029151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:18.029151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:18.029151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:18.029151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:18.029151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:18.029151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:18.029151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:18.030073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:18.030073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:18.030073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:18.030073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:18.030073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:18.030073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:18.030073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:18.030073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:18.030073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:18.030073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:18.030073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:18.030073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:18.030073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:18.030073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:18.030073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:18.030073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:18.030857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:18.030857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:18.030857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:18.030857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:18.030857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:18.030857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:18.030857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:18.030857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:18.030857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:18.030857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:18.030857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:18.030857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:18.030857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 84/173] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:18.032303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8da35c-c0b3-4f51-a89c-94fd370ea4a1 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:18.032847 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8da35c-c0b3-4f51-a89c-94fd370ea4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:18.033328 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8da35c-c0b3-4f51-a89c-94fd370ea4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:18.071647 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-1e8da35c-c0b3-4f51-a89c-94fd370ea4a1 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:18.072925 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-1e8da35c-c0b3-4f51-a89c-94fd370ea4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:18.108873 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6763d104-468b-44ba-98d2-b4e856a23991 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:18.110752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6763d104-468b-44ba-98d2-b4e856a23991 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:18.190960 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 85/174] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:18.192002 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8da35c-c0b3-4f51-a89c-94fd370ea4a1 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:18.205269 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:18.210212 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:18.258951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 90/175] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:18.259827 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763d104-468b-44ba-98d2-b4e856a23991 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:18.322885 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8da35c-c0b3-4f51-a89c-94fd370ea4a1 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:18.365883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af8400-18c6-410c-92fb-d43c5c2b8a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:18.366623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af8400-18c6-410c-92fb-d43c5c2b8a5c None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:18.366911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af8400-18c6-410c-92fb-d43c5c2b8a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:18.367191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af8400-18c6-410c-92fb-d43c5c2b8a5c None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:18.367921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af8400-18c6-410c-92fb-d43c5c2b8a5c None admin] RBAC: Adding query filter params (name=member) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:18.368230 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af8400-18c6-410c-92fb-d43c5c2b8a5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:18.369770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af8400-18c6-410c-92fb-d43c5c2b8a5c None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:18.376335 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763d104-468b-44ba-98d2-b4e856a23991 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:18.379303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 86/176] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:18.381557 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab76dda-70e2-4d37-8370-6022a22532ac None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:18.381845 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab76dda-70e2-4d37-8370-6022a22532ac None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:18.382121 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab76dda-70e2-4d37-8370-6022a22532ac None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:18.383391 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 87/177] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:18.384278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab76dda-70e2-4d37-8370-6022a22532ac None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:18.414621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df6544-21f0-41f5-8462-3279cb2d9a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:18.415284 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df6544-21f0-41f5-8462-3279cb2d9a60 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:18.415610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df6544-21f0-41f5-8462-3279cb2d9a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:18.415888 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df6544-21f0-41f5-8462-3279cb2d9a60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:18.427820 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df6544-21f0-41f5-8462-3279cb2d9a60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:18.434036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-f8df6544-21f0-41f5-8462-3279cb2d9a60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:22:18.434036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:18.434036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:18.434036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:18.434036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:18.434036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:18.434036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:18.434036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:18.434036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:18.434036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:18.434036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:18.434036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:18.434036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:18.434036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:18.434036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:18.434036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:18.434938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:18.434938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:18.434938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:18.434938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:18.434938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:18.434938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:18.434938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:18.434938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:18.434938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:18.434938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:18.434938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:18.434938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:18.434938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:18.434938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:18.434938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:18.434938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:18.436296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:18.436296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:18.436296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:18.436296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:18.436296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:18.436296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:18.436296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:18.436296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:18.436296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:18.436296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:18.436296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:22:18.436296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:18.436296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 91/178] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:22:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:18.436965 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f40dd8-fded-4c4e-bba7-3d7c8bf2c45c None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:18.436965 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f40dd8-fded-4c4e-bba7-3d7c8bf2c45c None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:18.437344 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f40dd8-fded-4c4e-bba7-3d7c8bf2c45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:18.457414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-66f40dd8-fded-4c4e-bba7-3d7c8bf2c45c None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:18.458613 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-66f40dd8-fded-4c4e-bba7-3d7c8bf2c45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:18.461818 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:18.467553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:18.503884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab76dda-70e2-4d37-8370-6022a22532ac None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:18.537247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 92/179] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:18.538152 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f40dd8-fded-4c4e-bba7-3d7c8bf2c45c None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:18.546734 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704d245-aef4-4dbe-b451-3a58a0e36657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:18.547504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704d245-aef4-4dbe-b451-3a58a0e36657 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:18.547789 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704d245-aef4-4dbe-b451-3a58a0e36657 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:18.548066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704d245-aef4-4dbe-b451-3a58a0e36657 None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:18.549282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704d245-aef4-4dbe-b451-3a58a0e36657 None admin] RBAC: Authorizing `identity:get_project(project_id=5633253a5837491c995b7f11753765f8)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:18.550855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704d245-aef4-4dbe-b451-3a58a0e36657 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:18.552650 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 88/180] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:18 2022] GET /identity/v3/projects/5633253a5837491c995b7f11753765f8 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:18.554668 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704d245-aef4-4dbe-b451-3a58a0e36657 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:18.648449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f40dd8-fded-4c4e-bba7-3d7c8bf2c45c None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:18.663213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704d245-aef4-4dbe-b451-3a58a0e36657 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:18.685444 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f203b2-0c2f-49a6-a394-84bc3df0de37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:18.686031 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f203b2-0c2f-49a6-a394-84bc3df0de37 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:18.686316 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f203b2-0c2f-49a6-a394-84bc3df0de37 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:18.686596 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f203b2-0c2f-49a6-a394-84bc3df0de37 None admin] PATH_INFO: `/v3/users/dcba272d6ea54652b4d3c43cf492f3e7` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:18.687831 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f203b2-0c2f-49a6-a394-84bc3df0de37 None admin] RBAC: Authorizing `identity:get_user(user_id=dcba272d6ea54652b4d3c43cf492f3e7)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:18.689318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f203b2-0c2f-49a6-a394-84bc3df0de37 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:18.690932 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 93/181] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:18 2022] GET /identity/v3/users/dcba272d6ea54652b4d3c43cf492f3e7 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:18.691777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f203b2-0c2f-49a6-a394-84bc3df0de37 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:18.703411 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:18.704260 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc464a-8614-4db1-bd4c-937ede37f2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:18.705090 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc464a-8614-4db1-bd4c-937ede37f2fd None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:18.705502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc464a-8614-4db1-bd4c-937ede37f2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:18.705780 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc464a-8614-4db1-bd4c-937ede37f2fd None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:18.706542 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc464a-8614-4db1-bd4c-937ede37f2fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:18.706848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc464a-8614-4db1-bd4c-937ede37f2fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:18.708229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc464a-8614-4db1-bd4c-937ede37f2fd None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:18.716944 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 89/182] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:18.721829 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc464a-8614-4db1-bd4c-937ede37f2fd None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:18.724875 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:18.799671 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f203b2-0c2f-49a6-a394-84bc3df0de37 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:18.826507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc464a-8614-4db1-bd4c-937ede37f2fd None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:18.831926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45bb5c-5c21-4ff6-9887-9ddb909f34fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:18.832540 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45bb5c-5c21-4ff6-9887-9ddb909f34fc None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:18.832816 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45bb5c-5c21-4ff6-9887-9ddb909f34fc None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:18.833183 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45bb5c-5c21-4ff6-9887-9ddb909f34fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:18.833956 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45bb5c-5c21-4ff6-9887-9ddb909f34fc None admin] RBAC: Adding query filter params (role.id=ca37e43082fa40c6a88b12680a0ca94b, scope.project.id=5633253a5837491c995b7f11753765f8, user.id=2682f5470265422cbaaa91dfd16e8ea6) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:18.834258 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45bb5c-5c21-4ff6-9887-9ddb909f34fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:18.835689 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45bb5c-5c21-4ff6-9887-9ddb909f34fc None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:18.841970 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 94/183] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:22:18 2022] GET /identity/v3/role_assignments?user.id=2682f5470265422cbaaa91dfd16e8ea6&scope.project.id=5633253a5837491c995b7f11753765f8&role.id=ca37e43082fa40c6a88b12680a0ca94b => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:18.843199 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b62105-f77d-4f8d-9e7a-5c04c01de8e8 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:18.843491 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b62105-f77d-4f8d-9e7a-5c04c01de8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:18.843769 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b62105-f77d-4f8d-9e7a-5c04c01de8e8 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:18.845000 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 95/184] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:18.849264 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b62105-f77d-4f8d-9e7a-5c04c01de8e8 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:18.878859 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7e0f4-3084-4ccd-a12d-4b2ca986024e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:18.879703 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7e0f4-3084-4ccd-a12d-4b2ca986024e None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:18.880057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7e0f4-3084-4ccd-a12d-4b2ca986024e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:18.880409 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7e0f4-3084-4ccd-a12d-4b2ca986024e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:18.892379 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7e0f4-3084-4ccd-a12d-4b2ca986024e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:18.899088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-cdb7e0f4-3084-4ccd-a12d-4b2ca986024e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:18.899088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:18.899088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:18.899088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:18.899088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:18.899088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:18.899088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:18.899088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:18.899088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:18.899088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:18.899088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:18.899088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:18.899088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:18.899088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:18.899088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:18.899088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:18.900062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:18.900062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:18.900062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:18.900062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:18.900062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:18.900062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:18.900062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:18.900062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:18.900062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:18.900062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:18.900062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:18.900062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:18.900062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:18.900062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:18.900062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:18.900062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:18.901301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:18.901301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:18.901301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:18.901301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:18.901301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:18.901301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:18.901301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:18.901301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:18.901301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:18.901301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:18.901301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:18.901301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:18.901301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 90/185] 10.222.0.74 () {56 vars in 1114 bytes} [Wed Jul 27 16:22:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:18.902314 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7e0f4-3084-4ccd-a12d-4b2ca986024e None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:18.935690 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:18.971957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:18.973163 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b62105-f77d-4f8d-9e7a-5c04c01de8e8 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:19.016876 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498d1eb-3800-4948-84ba-2a517d0b9191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:19.017566 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7e0f4-3084-4ccd-a12d-4b2ca986024e None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:19.017676 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498d1eb-3800-4948-84ba-2a517d0b9191 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:19.017821 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498d1eb-3800-4948-84ba-2a517d0b9191 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:19.018094 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498d1eb-3800-4948-84ba-2a517d0b9191 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:19.056498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d32f08-31c7-4515-aa5e-8d89d9181057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:19.057180 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d32f08-31c7-4515-aa5e-8d89d9181057 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:19.057468 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d32f08-31c7-4515-aa5e-8d89d9181057 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:19.057751 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d32f08-31c7-4515-aa5e-8d89d9181057 None admin] PATH_INFO: `/v3/users/6db877c4af704661a458e7ac01a8ff7b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:19.073367 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d32f08-31c7-4515-aa5e-8d89d9181057 None admin] RBAC: Authorizing `identity:get_user(user_id=6db877c4af704661a458e7ac01a8ff7b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:19.074861 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d32f08-31c7-4515-aa5e-8d89d9181057 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:19.077442 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 91/186] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:18 2022] GET /identity/v3/users/6db877c4af704661a458e7ac01a8ff7b => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:19.078712 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd75096-5d82-4a56-8f9a-03dfb342c047 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:19.079063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd75096-5d82-4a56-8f9a-03dfb342c047 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:19.079348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd75096-5d82-4a56-8f9a-03dfb342c047 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:19.080695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 92/187] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:19.081752 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05748c28-eea8-4c29-9e2b-d20c1ba67417 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:19.082043 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05748c28-eea8-4c29-9e2b-d20c1ba67417 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:19.082406 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05748c28-eea8-4c29-9e2b-d20c1ba67417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:19.098591 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498d1eb-3800-4948-84ba-2a517d0b9191 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:19.099593 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-05748c28-eea8-4c29-9e2b-d20c1ba67417 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:19.100160 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498d1eb-3800-4948-84ba-2a517d0b9191 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:19.100789 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-05748c28-eea8-4c29-9e2b-d20c1ba67417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:19.167676 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:19.168634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 93/188] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:19.170051 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05748c28-eea8-4c29-9e2b-d20c1ba67417 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:19.199366 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 96/189] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:19.199981 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498d1eb-3800-4948-84ba-2a517d0b9191 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:19.234836 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:19.284088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05748c28-eea8-4c29-9e2b-d20c1ba67417 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:19.299110 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498d1eb-3800-4948-84ba-2a517d0b9191 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:19.318183 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8de0b-3873-4036-92c8-aa6dc4e246b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:19.318803 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8de0b-3873-4036-92c8-aa6dc4e246b9 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:19.319089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8de0b-3873-4036-92c8-aa6dc4e246b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:19.319369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8de0b-3873-4036-92c8-aa6dc4e246b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:19.320086 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac8de0b-3873-4036-92c8-aa6dc4e246b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:19.320394 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac8de0b-3873-4036-92c8-aa6dc4e246b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:19.321829 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac8de0b-3873-4036-92c8-aa6dc4e246b9 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:19.330422 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 94/190] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:22:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:19.333528 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed39e74-821a-4e70-b8fb-1ca07b050d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:19.334153 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed39e74-821a-4e70-b8fb-1ca07b050d94 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:19.334253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f513a3-f6f4-4454-9a99-7755dd252435 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:19.334437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f513a3-f6f4-4454-9a99-7755dd252435 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:19.334532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed39e74-821a-4e70-b8fb-1ca07b050d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:19.334715 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f513a3-f6f4-4454-9a99-7755dd252435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:19.334808 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed39e74-821a-4e70-b8fb-1ca07b050d94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:19.351630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-c6f513a3-f6f4-4454-9a99-7755dd252435 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:19.352824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c6f513a3-f6f4-4454-9a99-7755dd252435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:19.375725 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:19.402643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed39e74-821a-4e70-b8fb-1ca07b050d94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:19.404273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed39e74-821a-4e70-b8fb-1ca07b050d94 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:19.419060 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 95/191] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:19.420504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f513a3-f6f4-4454-9a99-7755dd252435 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:19.463146 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:19.505664 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 97/192] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:19.506502 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed39e74-821a-4e70-b8fb-1ca07b050d94 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:19.518558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f513a3-f6f4-4454-9a99-7755dd252435 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:19.552040 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2009d65-e90e-4e69-9bc8-63655faa91ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:19.552679 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2009d65-e90e-4e69-9bc8-63655faa91ec None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:19.553063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2009d65-e90e-4e69-9bc8-63655faa91ec None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:19.553340 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2009d65-e90e-4e69-9bc8-63655faa91ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:19.554481 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2009d65-e90e-4e69-9bc8-63655faa91ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:19.555933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2009d65-e90e-4e69-9bc8-63655faa91ec None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:19.557551 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 96/193] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:22:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:19.558300 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2009d65-e90e-4e69-9bc8-63655faa91ec None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:19.581655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:19.612154 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed39e74-821a-4e70-b8fb-1ca07b050d94 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:19.645477 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672477e-fb0b-4deb-92ca-dc55faae46e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:19.646112 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672477e-fb0b-4deb-92ca-dc55faae46e4 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:19.646390 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672477e-fb0b-4deb-92ca-dc55faae46e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:19.646668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672477e-fb0b-4deb-92ca-dc55faae46e4 None admin] PATH_INFO: `/v3/projects/e33f17fdda044f03b50c7ac98a11eed9` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:19.647815 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672477e-fb0b-4deb-92ca-dc55faae46e4 None admin] RBAC: Authorizing `identity:get_project(project_id=e33f17fdda044f03b50c7ac98a11eed9)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:19.649308 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672477e-fb0b-4deb-92ca-dc55faae46e4 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:19.650955 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 98/194] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:19 2022] GET /identity/v3/projects/e33f17fdda044f03b50c7ac98a11eed9 => generated 312 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:19.651705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672477e-fb0b-4deb-92ca-dc55faae46e4 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:19.659974 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2009d65-e90e-4e69-9bc8-63655faa91ec None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:19.671993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:19.694963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d9ea41-0ab6-4129-adc9-05cfe0966d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:19.695599 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9ea41-0ab6-4129-adc9-05cfe0966d0d None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:19.695874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9ea41-0ab6-4129-adc9-05cfe0966d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:19.696141 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9ea41-0ab6-4129-adc9-05cfe0966d0d None admin] PATH_INFO: `/v3/users/6db877c4af704661a458e7ac01a8ff7b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:19.697614 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9ea41-0ab6-4129-adc9-05cfe0966d0d None admin] RBAC: Authorizing `identity:get_user(user_id=6db877c4af704661a458e7ac01a8ff7b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:19.699116 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9ea41-0ab6-4129-adc9-05cfe0966d0d None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:19.701125 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 97/195] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:19 2022] GET /identity/v3/users/6db877c4af704661a458e7ac01a8ff7b => generated 283 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:19.701725 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d9ea41-0ab6-4129-adc9-05cfe0966d0d None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:19.756156 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672477e-fb0b-4deb-92ca-dc55faae46e4 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:19.789240 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab2089-c87a-4ba2-830d-043426dc1334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:19.789874 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab2089-c87a-4ba2-830d-043426dc1334 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:19.790152 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab2089-c87a-4ba2-830d-043426dc1334 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:19.790420 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab2089-c87a-4ba2-830d-043426dc1334 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:19.798552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:19.798959 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d9ea41-0ab6-4129-adc9-05cfe0966d0d None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:19.801945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab2089-c87a-4ba2-830d-043426dc1334 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:19.807115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-38ab2089-c87a-4ba2-830d-043426dc1334 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 16:22:19.807115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:19.807115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:19.807115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:19.807115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:19.807115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:19.807115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:19.807115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:19.807115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:19.807115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:19.807115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:19.807115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:19.807115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:19.807115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:19.807115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:19.807115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:19.808006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:19.808006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:19.808006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:19.808006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:19.808006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:19.808006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:19.808006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:19.808006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:19.808006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:19.808006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:19.808006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:19.808006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:19.808006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:19.808006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:19.808006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:19.808006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:19.808788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:19.808788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:19.808788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:19.808788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:19.808788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:19.808788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:19.808788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:19.808788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:19.808788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:19.808788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:19.808788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 16:22:19.808788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:19.808788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 99/196] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:19.810199 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab2089-c87a-4ba2-830d-043426dc1334 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:19.837002 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f4a83-4744-4660-b25d-a0f73514217f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:19.837635 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f4a83-4744-4660-b25d-a0f73514217f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:19.837911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f4a83-4744-4660-b25d-a0f73514217f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:19.838188 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f4a83-4744-4660-b25d-a0f73514217f None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:19.839333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f4a83-4744-4660-b25d-a0f73514217f None admin] RBAC: Authorizing `identity:get_project(project_id=6a66a60a14e247e5bf2142da6f43cc44)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:19.840767 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f4a83-4744-4660-b25d-a0f73514217f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:19.842975 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 98/197] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:19 2022] GET /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:19.843718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f4a83-4744-4660-b25d-a0f73514217f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:19.889710 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:19.912688 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab2089-c87a-4ba2-830d-043426dc1334 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:19.942487 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f4a83-4744-4660-b25d-a0f73514217f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:19.946603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76deb47-6d03-4998-a6c0-eefcafd2fea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:19.947231 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76deb47-6d03-4998-a6c0-eefcafd2fea9 None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:19.947516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76deb47-6d03-4998-a6c0-eefcafd2fea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:19.947790 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76deb47-6d03-4998-a6c0-eefcafd2fea9 None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:19.948467 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76deb47-6d03-4998-a6c0-eefcafd2fea9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:19.949942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76deb47-6d03-4998-a6c0-eefcafd2fea9 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:19.969989 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 100/198] 10.222.0.74 () {60 vars in 1132 bytes} [Wed Jul 27 16:22:19 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:22:19.970798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76deb47-6d03-4998-a6c0-eefcafd2fea9 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:19.977313 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92107fca-4ee3-4474-a0f5-27fea2e087b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:19.978280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92107fca-4ee3-4474-a0f5-27fea2e087b0 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:19.978701 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92107fca-4ee3-4474-a0f5-27fea2e087b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:19.979132 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92107fca-4ee3-4474-a0f5-27fea2e087b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:19.980203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92107fca-4ee3-4474-a0f5-27fea2e087b0 None admin] RBAC: Adding query filter params (role.id=8de48592893c43789e1d40158943fd52, scope.project.id=e33f17fdda044f03b50c7ac98a11eed9, user.id=dcba272d6ea54652b4d3c43cf492f3e7) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:19.980819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92107fca-4ee3-4474-a0f5-27fea2e087b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:19.982896 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92107fca-4ee3-4474-a0f5-27fea2e087b0 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:19.994484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 99/199] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:22:19 2022] GET /identity/v3/role_assignments?user.id=dcba272d6ea54652b4d3c43cf492f3e7&scope.project.id=e33f17fdda044f03b50c7ac98a11eed9&role.id=8de48592893c43789e1d40158943fd52 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:19.994484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92107fca-4ee3-4474-a0f5-27fea2e087b0 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:20.056813 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:20.115630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76deb47-6d03-4998-a6c0-eefcafd2fea9 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:20.159983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e877b-92ae-4194-829f-c331095704e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:20.160615 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e877b-92ae-4194-829f-c331095704e9 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:20.160978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e877b-92ae-4194-829f-c331095704e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:20.161551 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e877b-92ae-4194-829f-c331095704e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:20.173996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92107fca-4ee3-4474-a0f5-27fea2e087b0 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:20.189969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:20.229046 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b874a51-c32d-4ac3-9885-9f918d4d592f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:20.231471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b874a51-c32d-4ac3-9885-9f918d4d592f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:20.231471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b874a51-c32d-4ac3-9885-9f918d4d592f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:20.231471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b874a51-c32d-4ac3-9885-9f918d4d592f None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:20.231471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b874a51-c32d-4ac3-9885-9f918d4d592f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:20.231471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b874a51-c32d-4ac3-9885-9f918d4d592f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:20.233792 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b874a51-c32d-4ac3-9885-9f918d4d592f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:20.246536 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 100/200] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:20.247196 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b874a51-c32d-4ac3-9885-9f918d4d592f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:20.255607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5e877b-92ae-4194-829f-c331095704e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:20.257156 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5e877b-92ae-4194-829f-c331095704e9 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:20.326521 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:20.374783 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b874a51-c32d-4ac3-9885-9f918d4d592f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:20.384486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 101/201] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:20.385870 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e877b-92ae-4194-829f-c331095704e9 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:20.417531 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83090e-153f-40d2-8ffc-b675e6ac2c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:20.418475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83090e-153f-40d2-8ffc-b675e6ac2c35 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:20.418755 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83090e-153f-40d2-8ffc-b675e6ac2c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:20.419030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83090e-153f-40d2-8ffc-b675e6ac2c35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:20.420442 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83090e-153f-40d2-8ffc-b675e6ac2c35 None admin] RBAC: Adding query filter params (role.id=ca37e43082fa40c6a88b12680a0ca94b, scope.project.id=6a66a60a14e247e5bf2142da6f43cc44, user.id=6db877c4af704661a458e7ac01a8ff7b) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:20.420748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83090e-153f-40d2-8ffc-b675e6ac2c35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:20.422211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83090e-153f-40d2-8ffc-b675e6ac2c35 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:20.427900 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 101/202] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:22:20 2022] GET /identity/v3/role_assignments?user.id=6db877c4af704661a458e7ac01a8ff7b&scope.project.id=6a66a60a14e247e5bf2142da6f43cc44&role.id=ca37e43082fa40c6a88b12680a0ca94b => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:20.429008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83090e-153f-40d2-8ffc-b675e6ac2c35 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:20.506571 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:20.554206 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e877b-92ae-4194-829f-c331095704e9 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:20.603292 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a39f13-9136-4759-8431-e14642324b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:20.603877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a39f13-9136-4759-8431-e14642324b5d None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:20.604151 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a39f13-9136-4759-8431-e14642324b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:20.604471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a39f13-9136-4759-8431-e14642324b5d None admin] PATH_INFO: `/v3/users/db74378449764a4ea8aa811cf24e5581` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:20.606457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a39f13-9136-4759-8431-e14642324b5d None admin] RBAC: Authorizing `identity:get_user(user_id=db74378449764a4ea8aa811cf24e5581)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:20.608591 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a39f13-9136-4759-8431-e14642324b5d None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:20.613835 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 102/203] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:20 2022] GET /identity/v3/users/db74378449764a4ea8aa811cf24e5581 => generated 289 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:20.620140 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a39f13-9136-4759-8431-e14642324b5d None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:20.633966 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83090e-153f-40d2-8ffc-b675e6ac2c35 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:20.670669 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:20.696594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297dab3-f6b6-461c-8540-dfe16c5369b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:20.697557 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297dab3-f6b6-461c-8540-dfe16c5369b3 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:20.697981 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297dab3-f6b6-461c-8540-dfe16c5369b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:20.698398 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297dab3-f6b6-461c-8540-dfe16c5369b3 None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:20.699722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a297dab3-f6b6-461c-8540-dfe16c5369b3 None admin] RBAC: Authorizing `identity:get_project(project_id=6a66a60a14e247e5bf2142da6f43cc44)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:20.703407 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a297dab3-f6b6-461c-8540-dfe16c5369b3 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:20.705274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 102/204] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:20 2022] GET /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44 => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:20.716928 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297dab3-f6b6-461c-8540-dfe16c5369b3 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:20.846415 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a39f13-9136-4759-8431-e14642324b5d None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:20.860652 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297dab3-f6b6-461c-8540-dfe16c5369b3 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:20.863365 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:20.917825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa6d7b-b525-4398-b32e-f59b02f03594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:20.918454 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa6d7b-b525-4398-b32e-f59b02f03594 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:20.918733 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa6d7b-b525-4398-b32e-f59b02f03594 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:20.919003 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa6d7b-b525-4398-b32e-f59b02f03594 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:20.920322 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa6d7b-b525-4398-b32e-f59b02f03594 None admin] RBAC: Adding query filter params (role.id=70f3e09bc103410789de8e15c81d9cd3, scope.project.id=6a66a60a14e247e5bf2142da6f43cc44, user.id=6db877c4af704661a458e7ac01a8ff7b) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:20.920630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa6d7b-b525-4398-b32e-f59b02f03594 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:20.922114 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa6d7b-b525-4398-b32e-f59b02f03594 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:20.926825 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e4da6-47af-4099-ade8-afd653a2a791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:20.927445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e4da6-47af-4099-ade8-afd653a2a791 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:20.927720 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e4da6-47af-4099-ade8-afd653a2a791 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:20.927990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e4da6-47af-4099-ade8-afd653a2a791 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:20.945135 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 103/205] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:22:20 2022] GET /identity/v3/role_assignments?user.id=6db877c4af704661a458e7ac01a8ff7b&scope.project.id=6a66a60a14e247e5bf2142da6f43cc44&role.id=70f3e09bc103410789de8e15c81d9cd3 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:21.021208 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e4da6-47af-4099-ade8-afd653a2a791 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:21.022675 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e4da6-47af-4099-ade8-afd653a2a791 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:21.049082 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:21.135356 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 103/206] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 16:22:21.142031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa6d7b-b525-4398-b32e-f59b02f03594 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:21.286660 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa6d7b-b525-4398-b32e-f59b02f03594 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:21.331990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab42b95-465f-4951-bedc-e0213cecf991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:21.332932 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab42b95-465f-4951-bedc-e0213cecf991 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:21.333378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab42b95-465f-4951-bedc-e0213cecf991 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:21.333810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab42b95-465f-4951-bedc-e0213cecf991 None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:21.335159 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab42b95-465f-4951-bedc-e0213cecf991 None admin] RBAC: Authorizing `identity:get_project(project_id=5633253a5837491c995b7f11753765f8)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:21.336762 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab42b95-465f-4951-bedc-e0213cecf991 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:21.338623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 104/207] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:21 2022] GET /identity/v3/projects/5633253a5837491c995b7f11753765f8 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:21.345135 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e4da6-47af-4099-ade8-afd653a2a791 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:21.505692 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e4da6-47af-4099-ade8-afd653a2a791 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:21.530840 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:21.541830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab99715-ea05-4c22-83af-74e258dd6eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:21.542674 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab99715-ea05-4c22-83af-74e258dd6eef None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:21.542957 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab99715-ea05-4c22-83af-74e258dd6eef None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:21.543233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab99715-ea05-4c22-83af-74e258dd6eef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:21.544148 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab99715-ea05-4c22-83af-74e258dd6eef None admin] RBAC: Adding query filter params (role.id=1204be9fc6d7483fb28ba04dfa5b3f6e, scope.project.id=5633253a5837491c995b7f11753765f8, user.id=db74378449764a4ea8aa811cf24e5581) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:21.544360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab99715-ea05-4c22-83af-74e258dd6eef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:21.545836 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab99715-ea05-4c22-83af-74e258dd6eef None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:21.552155 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 104/208] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:22:21 2022] GET /identity/v3/role_assignments?user.id=db74378449764a4ea8aa811cf24e5581&scope.project.id=5633253a5837491c995b7f11753765f8&role.id=1204be9fc6d7483fb28ba04dfa5b3f6e => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:22.450686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97311953-6b78-443b-8e72-0b6a416cbd62 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:22.451184 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97311953-6b78-443b-8e72-0b6a416cbd62 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:22.451571 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97311953-6b78-443b-8e72-0b6a416cbd62 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:22.453397 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 105/209] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:22.458367 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4567f-1bba-4056-84aa-190da0983734 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:22.458777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4567f-1bba-4056-84aa-190da0983734 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:22.458938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4567f-1bba-4056-84aa-190da0983734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:22.480789 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-91e4567f-1bba-4056-84aa-190da0983734 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:22.482846 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-91e4567f-1bba-4056-84aa-190da0983734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:22.557137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 105/210] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:22.579617 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97311953-6b78-443b-8e72-0b6a416cbd62 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:22.618537 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:22.639812 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758b006-3ba1-451d-a6ad-d2871d97a5e9 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:22.640140 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758b006-3ba1-451d-a6ad-d2871d97a5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:22.640361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758b006-3ba1-451d-a6ad-d2871d97a5e9 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:22.641984 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 106/211] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:22:22.646701 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e815e9e-3e8c-441b-a156-11e9dce16ac0 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:22.646985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e815e9e-3e8c-441b-a156-11e9dce16ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:22.647261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e815e9e-3e8c-441b-a156-11e9dce16ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:22.666574 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-8e815e9e-3e8c-441b-a156-11e9dce16ac0 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:22.667743 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-8e815e9e-3e8c-441b-a156-11e9dce16ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:22.693918 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97311953-6b78-443b-8e72-0b6a416cbd62 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:22.729385 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22c957-ecdb-479b-b05a-cb645e572092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:22.730357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22c957-ecdb-479b-b05a-cb645e572092 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:22.730837 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22c957-ecdb-479b-b05a-cb645e572092 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:22.731254 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22c957-ecdb-479b-b05a-cb645e572092 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:22.738459 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 107/212] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:22.739903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0e8ec-2591-4107-8ca5-3db8d0487e3e None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:22.740193 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0e8ec-2591-4107-8ca5-3db8d0487e3e None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:22.740467 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0e8ec-2591-4107-8ca5-3db8d0487e3e None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:22.741875 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 108/213] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:22.744449 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22c957-ecdb-479b-b05a-cb645e572092 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:22.746959 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc828950-2b70-4738-833f-b2818d98ec57 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:22.747780 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc828950-2b70-4738-833f-b2818d98ec57 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:22.748058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc828950-2b70-4738-833f-b2818d98ec57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:22.751374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-0d22c957-ecdb-479b-b05a-cb645e572092 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:22.751374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:22.751374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:22.751374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:22.751374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:22.751374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:22.751374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:22.751374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:22.751374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:22.751374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:22.751374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:22.751374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:22.751374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:22.751374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:22.751374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:22.751374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:22.752258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:22.752258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:22.752258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:22.752258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:22.752258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:22.752258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:22.752258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:22.752258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:22.752258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:22.752258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:22.752258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:22.752258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:22.752258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:22.752258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:22.752258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:22.752258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:22.753058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:22.753058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:22.753058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:22.753058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:22.753058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:22.753058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:22.753058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:22.753058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:22.753058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:22.753058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:22.753058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:22.753058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:22.755139 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 106/214] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:22.760832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22c957-ecdb-479b-b05a-cb645e572092 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:22.767163 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-cc828950-2b70-4738-833f-b2818d98ec57 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:22.768346 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-cc828950-2b70-4738-833f-b2818d98ec57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:22.821745 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:22.846613 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 109/215] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:22.847715 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc828950-2b70-4738-833f-b2818d98ec57 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:22.873385 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:22.882008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22c957-ecdb-479b-b05a-cb645e572092 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:22.917340 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee0f6b-8b48-42ab-bf89-a027f6493f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:22.918303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee0f6b-8b48-42ab-bf89-a027f6493f73 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:22.918591 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee0f6b-8b48-42ab-bf89-a027f6493f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:22.919532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee0f6b-8b48-42ab-bf89-a027f6493f73 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:22.920255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee0f6b-8b48-42ab-bf89-a027f6493f73 None admin] RBAC: Adding query filter params (name=member) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:22.920563 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee0f6b-8b48-42ab-bf89-a027f6493f73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:22.922559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee0f6b-8b48-42ab-bf89-a027f6493f73 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:22.933844 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 107/216] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:22.935451 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee0f6b-8b48-42ab-bf89-a027f6493f73 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:22.973414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc828950-2b70-4738-833f-b2818d98ec57 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:23.010464 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48427f25-ce14-4833-a9e0-3cdf05730f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:23.011034 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48427f25-ce14-4833-a9e0-3cdf05730f06 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:23.011314 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48427f25-ce14-4833-a9e0-3cdf05730f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:23.011611 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48427f25-ce14-4833-a9e0-3cdf05730f06 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:23.012772 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48427f25-ce14-4833-a9e0-3cdf05730f06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:23.014254 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48427f25-ce14-4833-a9e0-3cdf05730f06 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:23.015851 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 110/217] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:22:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:23.016605 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48427f25-ce14-4833-a9e0-3cdf05730f06 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:23.052055 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee0f6b-8b48-42ab-bf89-a027f6493f73 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:23.074496 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:23.094688 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299ae91-0884-4882-b9aa-d05a21c3c47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:23.095436 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299ae91-0884-4882-b9aa-d05a21c3c47f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:23.095720 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299ae91-0884-4882-b9aa-d05a21c3c47f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:23.096020 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299ae91-0884-4882-b9aa-d05a21c3c47f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:23.106507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8299ae91-0884-4882-b9aa-d05a21c3c47f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:23.112415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-8299ae91-0884-4882-b9aa-d05a21c3c47f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:22:23.112415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:23.112415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:23.112415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:23.112415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:23.112415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:23.112415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:23.112415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:23.112415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:23.112415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:23.112415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:23.112415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:23.112415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:23.112415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:23.112415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:23.112415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:23.113293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:23.113293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:23.113293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:23.113293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:23.113293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:23.113293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:23.113293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:23.113293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:23.113293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:23.113293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:23.113293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:23.113293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:23.113293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:23.113293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:23.113293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:23.113293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:23.114079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:23.114079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:23.114079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:23.114079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:23.114079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:23.114079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:23.114079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:23.114079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:23.114079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:23.114079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:23.114079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:22:23.114079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:23.114079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 108/218] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:22:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:23.114726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299ae91-0884-4882-b9aa-d05a21c3c47f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:23.135389 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48427f25-ce14-4833-a9e0-3cdf05730f06 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:23.135972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:23.174272 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fc016-3a29-467f-977d-0676a9fdb4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:23.174920 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fc016-3a29-467f-977d-0676a9fdb4ea None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:23.175199 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fc016-3a29-467f-977d-0676a9fdb4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:23.175475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fc016-3a29-467f-977d-0676a9fdb4ea None admin] PATH_INFO: `/v3/users/19c767183b0d491491c66eb1d4a0b02f` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:23.190535 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834fc016-3a29-467f-977d-0676a9fdb4ea None admin] RBAC: Authorizing `identity:get_user(user_id=19c767183b0d491491c66eb1d4a0b02f)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:23.191990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834fc016-3a29-467f-977d-0676a9fdb4ea None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:23.193685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 111/219] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:23 2022] GET /identity/v3/users/19c767183b0d491491c66eb1d4a0b02f => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:23.194448 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fc016-3a29-467f-977d-0676a9fdb4ea None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:23.219726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299ae91-0884-4882-b9aa-d05a21c3c47f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:23.257757 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d16ab2-262a-4eec-b8fe-a09558f1d894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:23.258374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d16ab2-262a-4eec-b8fe-a09558f1d894 None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:23.258668 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d16ab2-262a-4eec-b8fe-a09558f1d894 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:23.258942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d16ab2-262a-4eec-b8fe-a09558f1d894 None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:23.259618 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d16ab2-262a-4eec-b8fe-a09558f1d894 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:23.261190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d16ab2-262a-4eec-b8fe-a09558f1d894 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:23.282887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 109/220] 10.222.0.74 () {60 vars in 1132 bytes} [Wed Jul 27 16:22:23 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:22:23.282887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd717e-3440-4515-b10c-5510e1164b74 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:23.282887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd717e-3440-4515-b10c-5510e1164b74 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:23.282887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd717e-3440-4515-b10c-5510e1164b74 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:23.286003 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 110/221] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:23.292633 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd717e-3440-4515-b10c-5510e1164b74 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:23.316446 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fc016-3a29-467f-977d-0676a9fdb4ea None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:23.319238 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:23.354714 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c879b76-1b88-4e89-b0a7-16a37635ee41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:23.355411 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c879b76-1b88-4e89-b0a7-16a37635ee41 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:23.355705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c879b76-1b88-4e89-b0a7-16a37635ee41 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:23.356200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c879b76-1b88-4e89-b0a7-16a37635ee41 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:23.356982 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c879b76-1b88-4e89-b0a7-16a37635ee41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:23.357284 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c879b76-1b88-4e89-b0a7-16a37635ee41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:23.358699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c879b76-1b88-4e89-b0a7-16a37635ee41 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:23.368260 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 112/222] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:23.369985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6578a-9798-454d-8b4d-abeaa75b22cb None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:23.370533 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6578a-9798-454d-8b4d-abeaa75b22cb None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:23.371037 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6578a-9798-454d-8b4d-abeaa75b22cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:23.391546 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:23.392655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-e2a6578a-9798-454d-8b4d-abeaa75b22cb None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:23.393965 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e2a6578a-9798-454d-8b4d-abeaa75b22cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:23.421262 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd717e-3440-4515-b10c-5510e1164b74 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:23.460803 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5c34e-e8e9-44fe-a6d0-e15f1d0e25a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:23.462713 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5c34e-e8e9-44fe-a6d0-e15f1d0e25a2 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:23.463244 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5c34e-e8e9-44fe-a6d0-e15f1d0e25a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:23.463664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5c34e-e8e9-44fe-a6d0-e15f1d0e25a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:23.473850 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 113/223] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:23.477857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6578a-9798-454d-8b4d-abeaa75b22cb None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:23.547965 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5c34e-e8e9-44fe-a6d0-e15f1d0e25a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:23.549870 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5c34e-e8e9-44fe-a6d0-e15f1d0e25a2 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:23.603862 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:23.608721 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6578a-9798-454d-8b4d-abeaa75b22cb None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:23.654207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:23.657102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270e34e-5b42-46cd-a219-44e7e3d00e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:23.657755 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270e34e-5b42-46cd-a219-44e7e3d00e3a None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:23.658041 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270e34e-5b42-46cd-a219-44e7e3d00e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:23.658313 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270e34e-5b42-46cd-a219-44e7e3d00e3a None admin] PATH_INFO: `/v3/users/6db877c4af704661a458e7ac01a8ff7b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:23.659691 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d270e34e-5b42-46cd-a219-44e7e3d00e3a None admin] RBAC: Authorizing `identity:get_user(user_id=6db877c4af704661a458e7ac01a8ff7b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:23.661159 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d270e34e-5b42-46cd-a219-44e7e3d00e3a None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:23.662937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 114/224] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:23 2022] GET /identity/v3/users/6db877c4af704661a458e7ac01a8ff7b => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:23.664207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270e34e-5b42-46cd-a219-44e7e3d00e3a None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:23.693487 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 111/225] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:23.694966 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5c34e-e8e9-44fe-a6d0-e15f1d0e25a2 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:23.803152 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270e34e-5b42-46cd-a219-44e7e3d00e3a None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:23.831114 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5c34e-e8e9-44fe-a6d0-e15f1d0e25a2 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:23.868445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0a110-73ba-4c30-8ce4-3defb3607cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:23.869208 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0a110-73ba-4c30-8ce4-3defb3607cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:23.869646 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0a110-73ba-4c30-8ce4-3defb3607cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:23.869916 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0a110-73ba-4c30-8ce4-3defb3607cf1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:23.874517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a221995-7705-4a01-95fc-f88a23c5d829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:23.875438 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a221995-7705-4a01-95fc-f88a23c5d829 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:23.876077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a221995-7705-4a01-95fc-f88a23c5d829 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:23.876710 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a221995-7705-4a01-95fc-f88a23c5d829 None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:23.878467 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a221995-7705-4a01-95fc-f88a23c5d829 None admin] RBAC: Authorizing `identity:get_project(project_id=6a66a60a14e247e5bf2142da6f43cc44)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:23.880615 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a221995-7705-4a01-95fc-f88a23c5d829 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:23.882750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 112/226] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:23 2022] GET /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:23.883765 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0a110-73ba-4c30-8ce4-3defb3607cf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:23.884363 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a221995-7705-4a01-95fc-f88a23c5d829 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:23.890226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-70c0a110-73ba-4c30-8ce4-3defb3607cf1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:23.890226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:23.890226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:23.890226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:23.890226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:23.890226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:23.890226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:23.890226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:23.890226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:23.890226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:23.890226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:23.890226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:23.890226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:23.890226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:23.890226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:23.890226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:23.891111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:23.891111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:23.891111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:23.891111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:23.891111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:23.891111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:23.891111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:23.891111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:23.891111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:23.891111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:23.891111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:23.891111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:23.891111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:23.891111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:23.891111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:23.891111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:23.891896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:23.891896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:23.891896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:23.891896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:23.891896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:23.891896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:23.891896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:23.891896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:23.891896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:23.891896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:23.891896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:23.891896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:23.896064 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 115/227] 10.222.0.74 () {56 vars in 1114 bytes} [Wed Jul 27 16:22:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:23.896868 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0a110-73ba-4c30-8ce4-3defb3607cf1 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:23.951995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:23.953298 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:24.001498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a221995-7705-4a01-95fc-f88a23c5d829 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:24.037732 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0a110-73ba-4c30-8ce4-3defb3607cf1 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:24.074964 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdbb2c4-8981-44ff-94d6-eac235ce4317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:24.076304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbb2c4-8981-44ff-94d6-eac235ce4317 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:24.076695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbb2c4-8981-44ff-94d6-eac235ce4317 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:24.077187 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbb2c4-8981-44ff-94d6-eac235ce4317 None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:24.079108 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdbb2c4-8981-44ff-94d6-eac235ce4317 None admin] RBAC: Authorizing `identity:get_project(project_id=6a66a60a14e247e5bf2142da6f43cc44)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:24.080367 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e240661-ca68-41c6-a631-d4165107c617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:24.081243 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e240661-ca68-41c6-a631-d4165107c617 None admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:24.081680 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e240661-ca68-41c6-a631-d4165107c617 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:24.082094 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e240661-ca68-41c6-a631-d4165107c617 None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44/users/6db877c4af704661a458e7ac01a8ff7b/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:24.084580 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e240661-ca68-41c6-a631-d4165107c617 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a66a60a14e247e5bf2142da6f43cc44, user_id=6db877c4af704661a458e7ac01a8ff7b, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:24.084994 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdbb2c4-8981-44ff-94d6-eac235ce4317 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:24.086439 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e240661-ca68-41c6-a631-d4165107c617 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:24.087151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 113/228] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:23 2022] GET /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:24.087873 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdbb2c4-8981-44ff-94d6-eac235ce4317 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:24.115103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 116/229] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:22:23 2022] PUT /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44/users/6db877c4af704661a458e7ac01a8ff7b/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:22:24.116459 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e240661-ca68-41c6-a631-d4165107c617 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:24.220665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdbb2c4-8981-44ff-94d6-eac235ce4317 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:24.234411 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:24.270113 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc9b93-ab43-4080-94dd-7f4ad3505c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:24.271604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc9b93-ab43-4080-94dd-7f4ad3505c49 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:24.272231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc9b93-ab43-4080-94dd-7f4ad3505c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:24.273213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc9b93-ab43-4080-94dd-7f4ad3505c49 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:24.274372 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc9b93-ab43-4080-94dd-7f4ad3505c49 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:24.275911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc9b93-ab43-4080-94dd-7f4ad3505c49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:24.280810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc9b93-ab43-4080-94dd-7f4ad3505c49 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:24.296348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 114/230] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:22:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:24.302134 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc9b93-ab43-4080-94dd-7f4ad3505c49 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:24.345431 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:24.366668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e240661-ca68-41c6-a631-d4165107c617 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:24.409675 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abace9-0d50-418a-87bc-332cd58e4d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:24.410407 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abace9-0d50-418a-87bc-332cd58e4d9f None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:24.410756 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abace9-0d50-418a-87bc-332cd58e4d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:24.411119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abace9-0d50-418a-87bc-332cd58e4d9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:24.412588 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3abace9-0d50-418a-87bc-332cd58e4d9f None admin] RBAC: Adding query filter params (role.id=8de48592893c43789e1d40158943fd52, scope.project.id=6a66a60a14e247e5bf2142da6f43cc44, user.id=19c767183b0d491491c66eb1d4a0b02f) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:24.413019 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3abace9-0d50-418a-87bc-332cd58e4d9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:24.414651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3abace9-0d50-418a-87bc-332cd58e4d9f None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:24.423136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 117/231] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:22:24 2022] GET /identity/v3/role_assignments?user.id=19c767183b0d491491c66eb1d4a0b02f&scope.project.id=6a66a60a14e247e5bf2142da6f43cc44&role.id=8de48592893c43789e1d40158943fd52 => generated 259 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:24.436918 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc9b93-ab43-4080-94dd-7f4ad3505c49 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:24.476681 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bfb15-832d-46d2-8a26-bd604280b46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:24.477509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bfb15-832d-46d2-8a26-bd604280b46d None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:24.477674 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bfb15-832d-46d2-8a26-bd604280b46d None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:24.478585 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bfb15-832d-46d2-8a26-bd604280b46d None admin] PATH_INFO: `/v3/users/6db877c4af704661a458e7ac01a8ff7b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:24.479958 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581bfb15-832d-46d2-8a26-bd604280b46d None admin] RBAC: Authorizing `identity:get_user(user_id=6db877c4af704661a458e7ac01a8ff7b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:24.482019 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581bfb15-832d-46d2-8a26-bd604280b46d None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:24.483773 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 115/232] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:24 2022] GET /identity/v3/users/6db877c4af704661a458e7ac01a8ff7b => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:24.494052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abace9-0d50-418a-87bc-332cd58e4d9f None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:24.677277 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abace9-0d50-418a-87bc-332cd58e4d9f None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:24.713389 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eae608-51b5-4601-a4c2-ecd32d6b28f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:24.714055 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eae608-51b5-4601-a4c2-ecd32d6b28f7 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:24.714341 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eae608-51b5-4601-a4c2-ecd32d6b28f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:24.714610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eae608-51b5-4601-a4c2-ecd32d6b28f7 None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:24.715862 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eae608-51b5-4601-a4c2-ecd32d6b28f7 None admin] RBAC: Authorizing `identity:get_project(project_id=6a66a60a14e247e5bf2142da6f43cc44)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:24.717939 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eae608-51b5-4601-a4c2-ecd32d6b28f7 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:24.719725 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 118/233] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:24 2022] GET /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44 => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:24.725779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bfb15-832d-46d2-8a26-bd604280b46d None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:24.799002 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:24.877253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bfb15-832d-46d2-8a26-bd604280b46d None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:24.915532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6429f-8e34-4c50-b769-0fd7ea1e7e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:24.916455 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6429f-8e34-4c50-b769-0fd7ea1e7e34 None admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:24.916907 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6429f-8e34-4c50-b769-0fd7ea1e7e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:24.917374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6429f-8e34-4c50-b769-0fd7ea1e7e34 None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44/users/6db877c4af704661a458e7ac01a8ff7b/roles/70f3e09bc103410789de8e15c81d9cd3` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:24.919848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da6429f-8e34-4c50-b769-0fd7ea1e7e34 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a66a60a14e247e5bf2142da6f43cc44, user_id=6db877c4af704661a458e7ac01a8ff7b, role_id=70f3e09bc103410789de8e15c81d9cd3)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:24.921529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da6429f-8e34-4c50-b769-0fd7ea1e7e34 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:24.939598 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 116/234] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:22:24 2022] PUT /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44/users/6db877c4af704661a458e7ac01a8ff7b/roles/70f3e09bc103410789de8e15c81d9cd3 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:22:25.848119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d09d3-a8cb-49b1-8a5c-4f45f2561d24 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:25.849018 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d09d3-a8cb-49b1-8a5c-4f45f2561d24 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:25.849472 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d09d3-a8cb-49b1-8a5c-4f45f2561d24 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:25.851206 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 119/235] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:25.856628 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f0fb8-c842-436d-8617-cfb812c0d269 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:25.857460 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f0fb8-c842-436d-8617-cfb812c0d269 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:25.857944 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f0fb8-c842-436d-8617-cfb812c0d269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:25.878537 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-9c8f0fb8-c842-436d-8617-cfb812c0d269 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:25.880045 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9c8f0fb8-c842-436d-8617-cfb812c0d269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:25.951713 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 117/236] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:25.985296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d09d3-a8cb-49b1-8a5c-4f45f2561d24 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:26.005573 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:26.035210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3794633-5605-4a7a-bd37-72a7e097d47d None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:26.035875 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3794633-5605-4a7a-bd37-72a7e097d47d None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:26.036411 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3794633-5605-4a7a-bd37-72a7e097d47d None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:26.038371 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 118/237] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:26.048299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cffe9-f04e-4380-8c37-ce6377bc3245 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:26.048587 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cffe9-f04e-4380-8c37-ce6377bc3245 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:26.048867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cffe9-f04e-4380-8c37-ce6377bc3245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:26.089833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-026cffe9-f04e-4380-8c37-ce6377bc3245 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:26.091273 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-026cffe9-f04e-4380-8c37-ce6377bc3245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:26.116954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d09d3-a8cb-49b1-8a5c-4f45f2561d24 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:26.147703 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:26.161026 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce5a77-d705-45d6-9fc0-9e36446b89c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:26.162806 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce5a77-d705-45d6-9fc0-9e36446b89c2 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:26.163296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce5a77-d705-45d6-9fc0-9e36446b89c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:26.163774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce5a77-d705-45d6-9fc0-9e36446b89c2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:26.178166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce5a77-d705-45d6-9fc0-9e36446b89c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:26.185429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-9fce5a77-d705-45d6-9fc0-9e36446b89c2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 16:22:26.185429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:26.185429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:26.185429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:26.185429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:26.185429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:26.185429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:26.185429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:26.185429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:26.185429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:26.185429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:26.185429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:26.185429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:26.185429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:26.185429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:26.185429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:26.186282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:26.186282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:26.186282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:26.186282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:26.186282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:26.186282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:26.186282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:26.186282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:26.186282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:26.186282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:26.186282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:26.186282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:26.186282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:26.186282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:26.186282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:26.186282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:26.187053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:26.187053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:26.187053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:26.187053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:26.187053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:26.187053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:26.187053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:26.187053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:26.187053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:26.187053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:26.187053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 16:22:26.187053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:26.189733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 120/238] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:26.195505 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce5a77-d705-45d6-9fc0-9e36446b89c2 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:26.285707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:26.289863 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 119/239] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:26.312712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce5a77-d705-45d6-9fc0-9e36446b89c2 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:26.326212 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026cffe9-f04e-4380-8c37-ce6377bc3245 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:26.355200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc0c5e-c304-4c61-bc8a-caa14cc0cc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:26.356183 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc0c5e-c304-4c61-bc8a-caa14cc0cc2a None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:26.357275 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc0c5e-c304-4c61-bc8a-caa14cc0cc2a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:26.358002 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc0c5e-c304-4c61-bc8a-caa14cc0cc2a None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:26.358922 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc0c5e-c304-4c61-bc8a-caa14cc0cc2a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:26.359426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc0c5e-c304-4c61-bc8a-caa14cc0cc2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:26.361062 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc0c5e-c304-4c61-bc8a-caa14cc0cc2a None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:26.371423 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 121/240] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:26.382057 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc0c5e-c304-4c61-bc8a-caa14cc0cc2a None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:26.502656 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc0c5e-c304-4c61-bc8a-caa14cc0cc2a None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:26.542933 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45e937-c3e1-471a-95fd-8cbaf1d73128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:26.544384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45e937-c3e1-471a-95fd-8cbaf1d73128 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:26.544928 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45e937-c3e1-471a-95fd-8cbaf1d73128 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:26.545416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45e937-c3e1-471a-95fd-8cbaf1d73128 None admin] PATH_INFO: `/v3/users/26eed0d8b1874a2e8b8175d992043d06` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:26.556361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:26.566555 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45e937-c3e1-471a-95fd-8cbaf1d73128 None admin] RBAC: Authorizing `identity:get_user(user_id=26eed0d8b1874a2e8b8175d992043d06)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:26.568447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45e937-c3e1-471a-95fd-8cbaf1d73128 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:26.571143 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 122/241] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:26 2022] GET /identity/v3/users/26eed0d8b1874a2e8b8175d992043d06 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:26.576934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45e937-c3e1-471a-95fd-8cbaf1d73128 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:26.589555 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026cffe9-f04e-4380-8c37-ce6377bc3245 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:26.666596 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d647d43-2e25-4b69-8844-498ed513c611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:26.667168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d647d43-2e25-4b69-8844-498ed513c611 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:26.667448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d647d43-2e25-4b69-8844-498ed513c611 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:26.667721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d647d43-2e25-4b69-8844-498ed513c611 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:26.670088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d647d43-2e25-4b69-8844-498ed513c611 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:26.671563 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d647d43-2e25-4b69-8844-498ed513c611 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:26.673380 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 120/242] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:22:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:26.678437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d647d43-2e25-4b69-8844-498ed513c611 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:26.718213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45e937-c3e1-471a-95fd-8cbaf1d73128 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:26.746712 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:26.756066 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b601d7-d085-4275-abf0-76b92529ca96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:26.757051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b601d7-d085-4275-abf0-76b92529ca96 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:26.757537 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b601d7-d085-4275-abf0-76b92529ca96 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:26.758032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b601d7-d085-4275-abf0-76b92529ca96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:26.824702 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:26.832678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d647d43-2e25-4b69-8844-498ed513c611 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:26.832933 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b601d7-d085-4275-abf0-76b92529ca96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:26.834531 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b601d7-d085-4275-abf0-76b92529ca96 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:26.866293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f96100-9606-48f9-a276-ffa6fd0e8b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:26.866913 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f96100-9606-48f9-a276-ffa6fd0e8b29 None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:26.867228 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f96100-9606-48f9-a276-ffa6fd0e8b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:26.867654 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f96100-9606-48f9-a276-ffa6fd0e8b29 None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:26.868335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f96100-9606-48f9-a276-ffa6fd0e8b29 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:26.870326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f96100-9606-48f9-a276-ffa6fd0e8b29 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:26.892721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 121/243] 10.222.0.74 () {60 vars in 1132 bytes} [Wed Jul 27 16:22:26 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:22:26.903575 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d999367-ff65-473c-9ebc-aff815372d33 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:26.904127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d999367-ff65-473c-9ebc-aff815372d33 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:26.904421 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d999367-ff65-473c-9ebc-aff815372d33 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:26.906838 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 122/244] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:26.908387 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ee5f9-0e82-417c-8e03-7afb8a06fa73 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:26.908685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ee5f9-0e82-417c-8e03-7afb8a06fa73 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:26.909092 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ee5f9-0e82-417c-8e03-7afb8a06fa73 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:26.910899 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 123/245] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:26.914149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c581acd-4f4f-4e4a-b84d-917fb2dcab17 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:26.914443 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c581acd-4f4f-4e4a-b84d-917fb2dcab17 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:26.914774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c581acd-4f4f-4e4a-b84d-917fb2dcab17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:26.941341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-8c581acd-4f4f-4e4a-b84d-917fb2dcab17 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:26.943386 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-8c581acd-4f4f-4e4a-b84d-917fb2dcab17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:26.957924 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 123/246] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:26.959949 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0ad9b-245b-49a8-b168-4039fe44e328 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:26.960388 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0ad9b-245b-49a8-b168-4039fe44e328 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:26.960803 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0ad9b-245b-49a8-b168-4039fe44e328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:26.980997 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-49f0ad9b-245b-49a8-b168-4039fe44e328 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:26.982997 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-49f0ad9b-245b-49a8-b168-4039fe44e328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:27.052926 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 124/247] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:27.052926 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c581acd-4f4f-4e4a-b84d-917fb2dcab17 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:27.069333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:27.081654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 124/248] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:27.082363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0ad9b-245b-49a8-b168-4039fe44e328 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:27.121296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:27.185787 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c581acd-4f4f-4e4a-b84d-917fb2dcab17 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:27.215000 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0ad9b-245b-49a8-b168-4039fe44e328 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:27.226818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f7a4b-0fb4-49ff-9c5d-2eb3d8da957c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:27.227472 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f7a4b-0fb4-49ff-9c5d-2eb3d8da957c None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:27.227750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f7a4b-0fb4-49ff-9c5d-2eb3d8da957c None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:27.228024 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f7a4b-0fb4-49ff-9c5d-2eb3d8da957c None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:27.229318 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f7a4b-0fb4-49ff-9c5d-2eb3d8da957c None admin] RBAC: Authorizing `identity:get_project(project_id=6a66a60a14e247e5bf2142da6f43cc44)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:27.232159 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f7a4b-0fb4-49ff-9c5d-2eb3d8da957c None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:27.236330 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 125/249] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:27 2022] GET /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:27.237277 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f7a4b-0fb4-49ff-9c5d-2eb3d8da957c None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:27.258237 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99f392-ca65-4378-a923-68f661ce6e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:27.259532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99f392-ca65-4378-a923-68f661ce6e90 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:27.260115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99f392-ca65-4378-a923-68f661ce6e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:27.260720 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99f392-ca65-4378-a923-68f661ce6e90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:27.275325 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99f392-ca65-4378-a923-68f661ce6e90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:27.283519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-ab99f392-ca65-4378-a923-68f661ce6e90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:22:27.283519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:27.283519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:27.283519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:27.283519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:27.283519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:27.283519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:27.283519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:27.283519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:27.283519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:27.283519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:27.283519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:27.283519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:27.283519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:27.283519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:27.283519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:27.284575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:27.284575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:27.284575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:27.284575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:27.284575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:27.284575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:27.284575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:27.284575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:27.284575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:27.284575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:27.284575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:27.284575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:27.284575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:27.284575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:27.284575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:27.284575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:27.285481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:27.285481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:27.285481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:27.285481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:27.285481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:27.285481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:27.285481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:27.285481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:27.285481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:27.285481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:27.285481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:22:27.285481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:27.286379 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 125/250] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:22:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:27.287243 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99f392-ca65-4378-a923-68f661ce6e90 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:27.350225 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:27.369037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f7a4b-0fb4-49ff-9c5d-2eb3d8da957c None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:27.384178 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:27.405973 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99f392-ca65-4378-a923-68f661ce6e90 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:27.417782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa0c6b-96d7-4256-8557-496d08a70897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:27.418766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa0c6b-96d7-4256-8557-496d08a70897 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:27.419368 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa0c6b-96d7-4256-8557-496d08a70897 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:27.419698 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa0c6b-96d7-4256-8557-496d08a70897 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:27.441578 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa0c6b-96d7-4256-8557-496d08a70897 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:27.446638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-70fa0c6b-96d7-4256-8557-496d08a70897 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:27.446638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:27.446638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:27.446638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:27.446638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:27.446638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:27.446638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:27.446638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:27.446638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:27.446638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:27.446638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:27.446638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:27.446638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:27.446638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:27.446638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:27.446638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:27.447716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:27.447716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:27.447716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:27.447716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:27.447716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:27.447716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:27.447716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:27.447716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:27.447716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:27.447716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:27.447716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:27.447716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:27.447716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:27.447716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:27.447716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:27.447716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:27.448502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:27.448502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:27.448502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:27.448502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:27.448502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:27.448502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:27.448502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:27.448502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:27.448502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:27.448502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:27.448502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:27.448502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:27.448502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 126/251] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:27.449205 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa0c6b-96d7-4256-8557-496d08a70897 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:27.452827 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43515363-11be-4dfc-967b-472ac8636b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:27.454594 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43515363-11be-4dfc-967b-472ac8636b07 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:27.454879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43515363-11be-4dfc-967b-472ac8636b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:27.455152 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43515363-11be-4dfc-967b-472ac8636b07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:27.455952 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43515363-11be-4dfc-967b-472ac8636b07 None admin] RBAC: Adding query filter params (role.id=1204be9fc6d7483fb28ba04dfa5b3f6e, scope.project.id=6a66a60a14e247e5bf2142da6f43cc44, user.id=26eed0d8b1874a2e8b8175d992043d06) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:27.456312 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43515363-11be-4dfc-967b-472ac8636b07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:27.457921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43515363-11be-4dfc-967b-472ac8636b07 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:27.466739 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 126/252] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:22:27 2022] GET /identity/v3/role_assignments?user.id=26eed0d8b1874a2e8b8175d992043d06&scope.project.id=6a66a60a14e247e5bf2142da6f43cc44&role.id=1204be9fc6d7483fb28ba04dfa5b3f6e => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:27.467812 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43515363-11be-4dfc-967b-472ac8636b07 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:27.605023 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43515363-11be-4dfc-967b-472ac8636b07 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:27.654398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5983fc-0fe5-4375-9723-2db7341cad23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:27.655063 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5983fc-0fe5-4375-9723-2db7341cad23 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:27.655345 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5983fc-0fe5-4375-9723-2db7341cad23 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:27.655621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5983fc-0fe5-4375-9723-2db7341cad23 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:27.656361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5983fc-0fe5-4375-9723-2db7341cad23 None admin] RBAC: Adding query filter params (name=member) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:27.656662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5983fc-0fe5-4375-9723-2db7341cad23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:27.660919 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5983fc-0fe5-4375-9723-2db7341cad23 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:27.670441 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 127/253] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:27.672692 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e1a72-5b3c-4a36-a309-815a4b28973b None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:27.672692 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e1a72-5b3c-4a36-a309-815a4b28973b None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:27.672692 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e1a72-5b3c-4a36-a309-815a4b28973b None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:27.673838 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 128/254] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:27.679610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3ec5a-75aa-41ec-bb91-b43f685d1bcd None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:27.679891 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3ec5a-75aa-41ec-bb91-b43f685d1bcd None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:27.680167 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3ec5a-75aa-41ec-bb91-b43f685d1bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:27.695218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa0c6b-96d7-4256-8557-496d08a70897 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:27.703980 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-17a3ec5a-75aa-41ec-bb91-b43f685d1bcd None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:27.705260 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-17a3ec5a-75aa-41ec-bb91-b43f685d1bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:27.713875 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:27.746592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce58a8-f9a5-48ca-a341-89e5132175f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:27.747675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce58a8-f9a5-48ca-a341-89e5132175f2 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:27.747948 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce58a8-f9a5-48ca-a341-89e5132175f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:27.748214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce58a8-f9a5-48ca-a341-89e5132175f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:27.748942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce58a8-f9a5-48ca-a341-89e5132175f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:27.749244 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce58a8-f9a5-48ca-a341-89e5132175f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:27.756504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce58a8-f9a5-48ca-a341-89e5132175f2 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:27.773322 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 127/255] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:27.776678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce58a8-f9a5-48ca-a341-89e5132175f2 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:27.785881 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:27.804129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 129/256] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:27.805011 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3ec5a-75aa-41ec-bb91-b43f685d1bcd None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:27.939041 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce58a8-f9a5-48ca-a341-89e5132175f2 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:27.964072 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3ec5a-75aa-41ec-bb91-b43f685d1bcd None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:28.003028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1aec4-3b40-4c8b-906c-aba2bfbcdf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:28.004413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1aec4-3b40-4c8b-906c-aba2bfbcdf7e None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:28.005056 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1aec4-3b40-4c8b-906c-aba2bfbcdf7e None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:28.005539 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1aec4-3b40-4c8b-906c-aba2bfbcdf7e None admin] PATH_INFO: `/v3/users/6db877c4af704661a458e7ac01a8ff7b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:28.006984 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1aec4-3b40-4c8b-906c-aba2bfbcdf7e None admin] RBAC: Authorizing `identity:get_user(user_id=6db877c4af704661a458e7ac01a8ff7b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:28.008766 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1aec4-3b40-4c8b-906c-aba2bfbcdf7e None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:28.010926 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7f3fb-b7eb-42a9-8286-c355cf16d030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:28.011377 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 130/257] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:27 2022] GET /identity/v3/users/6db877c4af704661a458e7ac01a8ff7b => generated 283 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:28.011583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7f3fb-b7eb-42a9-8286-c355cf16d030 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:28.011861 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7f3fb-b7eb-42a9-8286-c355cf16d030 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:28.012142 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7f3fb-b7eb-42a9-8286-c355cf16d030 None admin] PATH_INFO: `/v3/users/19c767183b0d491491c66eb1d4a0b02f` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:28.013096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1aec4-3b40-4c8b-906c-aba2bfbcdf7e None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:28.017639 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c7f3fb-b7eb-42a9-8286-c355cf16d030 None admin] RBAC: Authorizing `identity:get_user(user_id=19c767183b0d491491c66eb1d4a0b02f)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:28.019078 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c7f3fb-b7eb-42a9-8286-c355cf16d030 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:28.020722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 128/258] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:27 2022] GET /identity/v3/users/19c767183b0d491491c66eb1d4a0b02f => generated 297 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:28.021514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7f3fb-b7eb-42a9-8286-c355cf16d030 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:28.026970 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:28.129831 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:28.173401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7f3fb-b7eb-42a9-8286-c355cf16d030 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:28.207955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41cd5f-246f-4d27-8232-d3a0b57b9198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:28.208581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41cd5f-246f-4d27-8232-d3a0b57b9198 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:28.208870 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41cd5f-246f-4d27-8232-d3a0b57b9198 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:28.209179 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41cd5f-246f-4d27-8232-d3a0b57b9198 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:28.230826 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1aec4-3b40-4c8b-906c-aba2bfbcdf7e None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:28.278075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41cd5f-246f-4d27-8232-d3a0b57b9198 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:28.280066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41cd5f-246f-4d27-8232-d3a0b57b9198 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:28.350477 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:28.381597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f1de8-1c5a-4087-a007-70696cce45b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:28.382329 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 129/259] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:28.382627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f1de8-1c5a-4087-a007-70696cce45b4 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:28.382919 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f1de8-1c5a-4087-a007-70696cce45b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:28.383162 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41cd5f-246f-4d27-8232-d3a0b57b9198 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:28.383265 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f1de8-1c5a-4087-a007-70696cce45b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:28.425989 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f1de8-1c5a-4087-a007-70696cce45b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:28.431398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-905f1de8-1c5a-4087-a007-70696cce45b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:28.431398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:28.431398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:28.431398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:28.431398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:28.431398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:28.431398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:28.431398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:28.431398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:28.431398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:28.431398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:28.431398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:28.431398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:28.431398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:28.431398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:28.431398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:28.432354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:28.432354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:28.432354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:28.432354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:28.432354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:28.432354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:28.432354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:28.432354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:28.432354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:28.432354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:28.432354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:28.432354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:28.432354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:28.432354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:28.432354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:28.432354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:28.433266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:28.433266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:28.433266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:28.433266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:28.433266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:28.433266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:28.433266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:28.433266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:28.433266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:28.433266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:28.433266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:28.433266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:28.441068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 131/260] 10.222.0.74 () {56 vars in 1114 bytes} [Wed Jul 27 16:22:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:28.442017 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f1de8-1c5a-4087-a007-70696cce45b4 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:28.513551 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41cd5f-246f-4d27-8232-d3a0b57b9198 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:28.569320 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:28.570136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da890da-9e3e-4379-a712-f3d67a0f354f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:28.570788 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da890da-9e3e-4379-a712-f3d67a0f354f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:28.571065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da890da-9e3e-4379-a712-f3d67a0f354f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:28.571337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da890da-9e3e-4379-a712-f3d67a0f354f None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:28.577346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da890da-9e3e-4379-a712-f3d67a0f354f None admin] RBAC: Authorizing `identity:get_project(project_id=6a66a60a14e247e5bf2142da6f43cc44)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:28.578813 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da890da-9e3e-4379-a712-f3d67a0f354f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:28.581083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 130/261] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:28 2022] GET /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:28.582042 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da890da-9e3e-4379-a712-f3d67a0f354f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:28.637571 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:28.657541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f1de8-1c5a-4087-a007-70696cce45b4 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:28.691151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da890da-9e3e-4379-a712-f3d67a0f354f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:28.724685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ead7a-2fe1-4651-8df1-d6b4695d49c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:28.725380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ead7a-2fe1-4651-8df1-d6b4695d49c3 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:28.725662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ead7a-2fe1-4651-8df1-d6b4695d49c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:28.726088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ead7a-2fe1-4651-8df1-d6b4695d49c3 None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:28.727508 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ead7a-2fe1-4651-8df1-d6b4695d49c3 None admin] RBAC: Authorizing `identity:get_project(project_id=6a66a60a14e247e5bf2142da6f43cc44)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:28.729111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ead7a-2fe1-4651-8df1-d6b4695d49c3 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:28.731155 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 132/262] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:28 2022] GET /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44 => generated 302 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:28.732548 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ead7a-2fe1-4651-8df1-d6b4695d49c3 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:28.761062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c9e51-43b3-485e-95c2-04368077acce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:28.761695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c9e51-43b3-485e-95c2-04368077acce None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:28.761979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c9e51-43b3-485e-95c2-04368077acce None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:28.762256 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c9e51-43b3-485e-95c2-04368077acce None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:28.762989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c9e51-43b3-485e-95c2-04368077acce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:28.763295 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c9e51-43b3-485e-95c2-04368077acce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:28.764691 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c9e51-43b3-485e-95c2-04368077acce None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:28.782657 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 131/263] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:22:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:28.783541 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c9e51-43b3-485e-95c2-04368077acce None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:28.926606 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ead7a-2fe1-4651-8df1-d6b4695d49c3 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:28.962174 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11d2d1-7621-4133-bb87-a00f3bd1762e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:28.962855 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11d2d1-7621-4133-bb87-a00f3bd1762e None admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:28.963140 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11d2d1-7621-4133-bb87-a00f3bd1762e None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:28.963418 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11d2d1-7621-4133-bb87-a00f3bd1762e None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44/users/19c767183b0d491491c66eb1d4a0b02f/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:28.965425 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11d2d1-7621-4133-bb87-a00f3bd1762e None admin] RBAC: Authorizing `identity:create_grant(project_id=6a66a60a14e247e5bf2142da6f43cc44, user_id=19c767183b0d491491c66eb1d4a0b02f, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:28.967945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11d2d1-7621-4133-bb87-a00f3bd1762e None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:28.983185 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 133/264] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:22:28 2022] PUT /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44/users/19c767183b0d491491c66eb1d4a0b02f/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:22:28.984200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11d2d1-7621-4133-bb87-a00f3bd1762e None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:28.985219 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:28.998951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c9e51-43b3-485e-95c2-04368077acce None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:29.003774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:29.151948 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11d2d1-7621-4133-bb87-a00f3bd1762e None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:29.170824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551931b1-b837-48b5-acfa-a400fb39b517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:29.171478 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551931b1-b837-48b5-acfa-a400fb39b517 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:29.171764 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551931b1-b837-48b5-acfa-a400fb39b517 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:29.172037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551931b1-b837-48b5-acfa-a400fb39b517 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:29.172844 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551931b1-b837-48b5-acfa-a400fb39b517 None admin] RBAC: Adding query filter params (role.id=ca37e43082fa40c6a88b12680a0ca94b, scope.project.id=6a66a60a14e247e5bf2142da6f43cc44, user.id=6db877c4af704661a458e7ac01a8ff7b) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:29.173173 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551931b1-b837-48b5-acfa-a400fb39b517 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:29.174635 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551931b1-b837-48b5-acfa-a400fb39b517 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:29.189561 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 132/265] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:22:28 2022] GET /identity/v3/role_assignments?user.id=6db877c4af704661a458e7ac01a8ff7b&scope.project.id=6a66a60a14e247e5bf2142da6f43cc44&role.id=ca37e43082fa40c6a88b12680a0ca94b => generated 608 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:29.218815 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef24aa86-d805-4d15-9ba1-aa761c32fdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:29.219868 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24aa86-d805-4d15-9ba1-aa761c32fdbc None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:29.220373 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24aa86-d805-4d15-9ba1-aa761c32fdbc None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:29.220843 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24aa86-d805-4d15-9ba1-aa761c32fdbc None admin] PATH_INFO: `/v3/users/6db877c4af704661a458e7ac01a8ff7b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:29.225905 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef24aa86-d805-4d15-9ba1-aa761c32fdbc None admin] RBAC: Authorizing `identity:get_user(user_id=6db877c4af704661a458e7ac01a8ff7b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:29.227804 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef24aa86-d805-4d15-9ba1-aa761c32fdbc None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:29.229920 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 134/266] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:28 2022] GET /identity/v3/users/6db877c4af704661a458e7ac01a8ff7b => generated 283 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:29.242138 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551931b1-b837-48b5-acfa-a400fb39b517 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:29.386028 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551931b1-b837-48b5-acfa-a400fb39b517 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:29.435978 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99bb1b-fad1-43ae-a205-63be6c382704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:29.436711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99bb1b-fad1-43ae-a205-63be6c382704 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:29.437278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99bb1b-fad1-43ae-a205-63be6c382704 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:29.437786 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99bb1b-fad1-43ae-a205-63be6c382704 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:29.490372 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:29.507004 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293df24d-c149-4e78-88a3-fcd92a44df87 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:29.507628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293df24d-c149-4e78-88a3-fcd92a44df87 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:29.508087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293df24d-c149-4e78-88a3-fcd92a44df87 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:29.513815 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 135/267] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:29.522390 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabac143-a22d-4270-9e88-fa0a0ee09d6e None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:29.522987 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabac143-a22d-4270-9e88-fa0a0ee09d6e None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:29.523475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabac143-a22d-4270-9e88-fa0a0ee09d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:29.564434 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99bb1b-fad1-43ae-a205-63be6c382704 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:29.565957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99bb1b-fad1-43ae-a205-63be6c382704 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:29.573977 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-aabac143-a22d-4270-9e88-fa0a0ee09d6e None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:29.575742 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-aabac143-a22d-4270-9e88-fa0a0ee09d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:29.679273 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 133/268] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:29.685891 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99bb1b-fad1-43ae-a205-63be6c382704 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:29.691969 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:29.709066 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 136/269] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:29.736672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73c5e7-0d64-439d-8235-1190e128324d None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:29.737004 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73c5e7-0d64-439d-8235-1190e128324d None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:29.737263 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73c5e7-0d64-439d-8235-1190e128324d None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:29.738695 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 137/270] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:22:29.743982 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386890b0-129d-4e8f-ae00-a07cea709cea None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:29.744204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386890b0-129d-4e8f-ae00-a07cea709cea None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:29.744471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386890b0-129d-4e8f-ae00-a07cea709cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:29.756367 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:29.769421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-386890b0-129d-4e8f-ae00-a07cea709cea None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:29.773236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-386890b0-129d-4e8f-ae00-a07cea709cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:29.796070 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99bb1b-fad1-43ae-a205-63be6c382704 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:29.833862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d063d-7944-461c-9584-2472663f497c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:29.834512 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d063d-7944-461c-9584-2472663f497c None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:29.834788 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d063d-7944-461c-9584-2472663f497c None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:29.835060 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d063d-7944-461c-9584-2472663f497c None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:29.836233 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d063d-7944-461c-9584-2472663f497c None admin] RBAC: Authorizing `identity:get_project(project_id=6a66a60a14e247e5bf2142da6f43cc44)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:29.837721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d063d-7944-461c-9584-2472663f497c None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:29.839382 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 134/271] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:29 2022] GET /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:29.840233 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d063d-7944-461c-9584-2472663f497c None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:29.867134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 138/272] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:29.869030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386890b0-129d-4e8f-ae00-a07cea709cea None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:29.975256 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d063d-7944-461c-9584-2472663f497c None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:30.016459 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fe2b4-283b-4e52-bbd0-7e2cbf2ca65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:30.017435 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386890b0-129d-4e8f-ae00-a07cea709cea None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:30.018211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fe2b4-283b-4e52-bbd0-7e2cbf2ca65b None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:30.018631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fe2b4-283b-4e52-bbd0-7e2cbf2ca65b None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:30.019040 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fe2b4-283b-4e52-bbd0-7e2cbf2ca65b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:30.025415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:30.039469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fe2b4-283b-4e52-bbd0-7e2cbf2ca65b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:30.046390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-889fe2b4-283b-4e52-bbd0-7e2cbf2ca65b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:30.046390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:30.046390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:30.046390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:30.046390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:30.046390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:30.046390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:30.046390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:30.046390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:30.046390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:30.046390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:30.046390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:30.046390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:30.046390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:30.046390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:30.046390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:30.047274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:30.047274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:30.047274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:30.047274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:30.047274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:30.047274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:30.047274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:30.047274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:30.047274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:30.047274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:30.047274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:30.047274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:30.047274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:30.047274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:30.047274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:30.047274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:30.048049 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:30.048049 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:30.048049 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:30.048049 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:30.048049 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:30.048049 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:30.048049 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:30.048049 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:30.048049 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:30.048049 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:30.048049 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:30.048049 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:30.050510 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 135/273] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:30.051471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fe2b4-283b-4e52-bbd0-7e2cbf2ca65b None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:30.056742 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:30.069005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369aab1-f848-401a-ac66-ba6a8c8a53d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:30.069915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369aab1-f848-401a-ac66-ba6a8c8a53d6 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:30.070200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369aab1-f848-401a-ac66-ba6a8c8a53d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:30.070477 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369aab1-f848-401a-ac66-ba6a8c8a53d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:30.071431 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369aab1-f848-401a-ac66-ba6a8c8a53d6 None admin] RBAC: Adding query filter params (role.id=70f3e09bc103410789de8e15c81d9cd3, scope.project.id=6a66a60a14e247e5bf2142da6f43cc44, user.id=6db877c4af704661a458e7ac01a8ff7b) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:30.071739 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369aab1-f848-401a-ac66-ba6a8c8a53d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:30.073848 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369aab1-f848-401a-ac66-ba6a8c8a53d6 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:30.081417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 139/274] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:22:29 2022] GET /identity/v3/role_assignments?user.id=6db877c4af704661a458e7ac01a8ff7b&scope.project.id=6a66a60a14e247e5bf2142da6f43cc44&role.id=70f3e09bc103410789de8e15c81d9cd3 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:30.082203 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369aab1-f848-401a-ac66-ba6a8c8a53d6 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:30.194277 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fe2b4-283b-4e52-bbd0-7e2cbf2ca65b None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:30.212576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369aab1-f848-401a-ac66-ba6a8c8a53d6 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:30.234538 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099ec79-0c74-46d3-b594-c70b3f8cc74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:30.235461 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099ec79-0c74-46d3-b594-c70b3f8cc74d None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:30.235737 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099ec79-0c74-46d3-b594-c70b3f8cc74d None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:30.236009 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099ec79-0c74-46d3-b594-c70b3f8cc74d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:30.237362 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099ec79-0c74-46d3-b594-c70b3f8cc74d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:30.239019 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099ec79-0c74-46d3-b594-c70b3f8cc74d None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:30.241287 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 136/275] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:22:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:30.246953 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf382261-4f4d-4b12-a188-ac496b2239b8 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:30.246953 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf382261-4f4d-4b12-a188-ac496b2239b8 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:30.246953 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf382261-4f4d-4b12-a188-ac496b2239b8 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:30.247881 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 137/276] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:30.248471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf382261-4f4d-4b12-a188-ac496b2239b8 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:30.251468 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f279a-28ff-4cc5-a999-448f65e4044b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:30.252282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f279a-28ff-4cc5-a999-448f65e4044b None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:30.252826 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f279a-28ff-4cc5-a999-448f65e4044b None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:30.253494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f279a-28ff-4cc5-a999-448f65e4044b None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:30.254368 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f279a-28ff-4cc5-a999-448f65e4044b None admin] RBAC: Adding query filter params (name=member) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:30.254671 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f279a-28ff-4cc5-a999-448f65e4044b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:30.256036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f279a-28ff-4cc5-a999-448f65e4044b None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:30.269054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 140/277] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:30.270980 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea2ce4-e4cc-4840-abec-450b37e333bb None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:30.271465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea2ce4-e4cc-4840-abec-450b37e333bb None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:30.271752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea2ce4-e4cc-4840-abec-450b37e333bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:30.296226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-edea2ce4-e4cc-4840-abec-450b37e333bb None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:30.297523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-edea2ce4-e4cc-4840-abec-450b37e333bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:30.323128 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:30.364244 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:30.373540 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf382261-4f4d-4b12-a188-ac496b2239b8 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:30.393457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 141/278] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:30.393457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea2ce4-e4cc-4840-abec-450b37e333bb None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:30.419633 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f84cdbc-0f6d-4338-abdb-107512f7c5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:30.420498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84cdbc-0f6d-4338-abdb-107512f7c5be None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:30.420955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84cdbc-0f6d-4338-abdb-107512f7c5be None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:30.421229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84cdbc-0f6d-4338-abdb-107512f7c5be None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:30.421905 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f84cdbc-0f6d-4338-abdb-107512f7c5be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:30.423342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f84cdbc-0f6d-4338-abdb-107512f7c5be None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:30.446905 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 138/279] 10.222.0.74 () {60 vars in 1132 bytes} [Wed Jul 27 16:22:30 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:22:30.454868 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f84cdbc-0f6d-4338-abdb-107512f7c5be None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:30.524958 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea2ce4-e4cc-4840-abec-450b37e333bb None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:30.574359 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c61228-78fd-4bc5-bead-2d94a4c86b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:30.575621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c61228-78fd-4bc5-bead-2d94a4c86b6d None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:30.575911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c61228-78fd-4bc5-bead-2d94a4c86b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:30.576190 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c61228-78fd-4bc5-bead-2d94a4c86b6d None admin] PATH_INFO: `/v3/users/d3f366d1181041a1a4fe50d790382201` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:30.585279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f84cdbc-0f6d-4338-abdb-107512f7c5be None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:30.595083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c61228-78fd-4bc5-bead-2d94a4c86b6d None admin] RBAC: Authorizing `identity:get_user(user_id=d3f366d1181041a1a4fe50d790382201)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:30.597033 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c61228-78fd-4bc5-bead-2d94a4c86b6d None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:30.603644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 142/280] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:30 2022] GET /identity/v3/users/d3f366d1181041a1a4fe50d790382201 => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:30.605459 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c61228-78fd-4bc5-bead-2d94a4c86b6d None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:30.614009 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:30.643069 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35171d-d6e9-4bf5-ba29-d206c047429f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:30.644111 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35171d-d6e9-4bf5-ba29-d206c047429f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:30.644591 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35171d-d6e9-4bf5-ba29-d206c047429f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:30.645106 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35171d-d6e9-4bf5-ba29-d206c047429f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:30.655577 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:30.658522 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c35171d-d6e9-4bf5-ba29-d206c047429f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:30.665729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-0c35171d-d6e9-4bf5-ba29-d206c047429f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 16:22:30.665729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:30.665729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:30.665729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:30.665729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:30.665729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:30.665729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:30.665729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:30.665729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:30.665729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:30.665729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:30.665729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:30.665729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:30.665729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:30.665729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:30.665729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:30.666622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:30.666622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:30.666622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:30.666622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:30.666622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:30.666622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:30.666622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:30.666622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:30.666622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:30.666622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:30.666622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:30.666622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:30.666622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:30.666622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:30.666622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:30.666622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:30.667401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:30.667401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:30.667401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:30.667401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:30.667401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:30.667401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:30.667401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:30.667401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:30.667401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:30.667401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:30.667401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 16:22:30.667401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:30.667401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 139/281] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:30.675160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35171d-d6e9-4bf5-ba29-d206c047429f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:30.731053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c61228-78fd-4bc5-bead-2d94a4c86b6d None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:30.770905 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d832de-f487-4911-8e8f-b3fc865bda97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:30.771575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d832de-f487-4911-8e8f-b3fc865bda97 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:30.771852 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d832de-f487-4911-8e8f-b3fc865bda97 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:30.772169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d832de-f487-4911-8e8f-b3fc865bda97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:30.773691 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d832de-f487-4911-8e8f-b3fc865bda97 None admin] RBAC: Adding query filter params (role.id=8de48592893c43789e1d40158943fd52, user.id=d3f366d1181041a1a4fe50d790382201, scope.system=all) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:30.774011 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d832de-f487-4911-8e8f-b3fc865bda97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:30.775510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d832de-f487-4911-8e8f-b3fc865bda97 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:30.781213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 143/282] 10.222.0.74 () {56 vars in 1308 bytes} [Wed Jul 27 16:22:30 2022] GET /identity/v3/role_assignments?user.id=d3f366d1181041a1a4fe50d790382201&scope.system=all&role.id=8de48592893c43789e1d40158943fd52 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:30.807284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35171d-d6e9-4bf5-ba29-d206c047429f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:30.853219 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63dab3a-d33d-44d9-93c9-d6c9f4c8a475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:30.853809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dab3a-d33d-44d9-93c9-d6c9f4c8a475 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:30.854087 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dab3a-d33d-44d9-93c9-d6c9f4c8a475 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:30.854356 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dab3a-d33d-44d9-93c9-d6c9f4c8a475 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:30.855085 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63dab3a-d33d-44d9-93c9-d6c9f4c8a475 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:30.855387 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63dab3a-d33d-44d9-93c9-d6c9f4c8a475 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:30.856875 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63dab3a-d33d-44d9-93c9-d6c9f4c8a475 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:30.866970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 140/283] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:30.874613 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d832de-f487-4911-8e8f-b3fc865bda97 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:30.987263 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d832de-f487-4911-8e8f-b3fc865bda97 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:30.989732 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:31.025006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd316ba2-c107-4274-b2e6-57b2238e6146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:31.025693 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd316ba2-c107-4274-b2e6-57b2238e6146 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:31.025967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd316ba2-c107-4274-b2e6-57b2238e6146 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:31.026309 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd316ba2-c107-4274-b2e6-57b2238e6146 None admin] PATH_INFO: `/v3/users/26eed0d8b1874a2e8b8175d992043d06` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:31.027529 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd316ba2-c107-4274-b2e6-57b2238e6146 None admin] RBAC: Authorizing `identity:get_user(user_id=26eed0d8b1874a2e8b8175d992043d06)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:31.029064 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd316ba2-c107-4274-b2e6-57b2238e6146 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:31.030974 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 144/284] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:30 2022] GET /identity/v3/users/26eed0d8b1874a2e8b8175d992043d06 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:31.036185 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63dab3a-d33d-44d9-93c9-d6c9f4c8a475 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:31.080833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:31.156006 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63dab3a-d33d-44d9-93c9-d6c9f4c8a475 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:31.208717 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ef671-e9d5-4ace-803a-aeea887688bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:31.209409 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ef671-e9d5-4ace-803a-aeea887688bc None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:31.209688 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ef671-e9d5-4ace-803a-aeea887688bc None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:31.209961 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ef671-e9d5-4ace-803a-aeea887688bc None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:31.213593 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ef671-e9d5-4ace-803a-aeea887688bc None admin] RBAC: Authorizing `identity:get_project(project_id=6a66a60a14e247e5bf2142da6f43cc44)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:31.215049 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ef671-e9d5-4ace-803a-aeea887688bc None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:31.217100 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 141/285] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:31 2022] GET /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:31.222724 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd316ba2-c107-4274-b2e6-57b2238e6146 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:31.341803 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd316ba2-c107-4274-b2e6-57b2238e6146 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:31.376339 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd799679-d1e0-4155-a35c-b330c899e870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:31.377094 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd799679-d1e0-4155-a35c-b330c899e870 None admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:31.377420 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd799679-d1e0-4155-a35c-b330c899e870 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:31.377698 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd799679-d1e0-4155-a35c-b330c899e870 None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44/users/26eed0d8b1874a2e8b8175d992043d06/roles/1204be9fc6d7483fb28ba04dfa5b3f6e` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:31.379761 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd799679-d1e0-4155-a35c-b330c899e870 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a66a60a14e247e5bf2142da6f43cc44, user_id=26eed0d8b1874a2e8b8175d992043d06, role_id=1204be9fc6d7483fb28ba04dfa5b3f6e)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:31.381229 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd799679-d1e0-4155-a35c-b330c899e870 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:31.402585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 145/286] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:22:31 2022] PUT /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44/users/26eed0d8b1874a2e8b8175d992043d06/roles/1204be9fc6d7483fb28ba04dfa5b3f6e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:22:31.765765 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a5a1b-d2f9-44be-a71a-d0f41b813865 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:31.766140 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a5a1b-d2f9-44be-a71a-d0f41b813865 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:31.766316 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a5a1b-d2f9-44be-a71a-d0f41b813865 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:31.767918 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 142/287] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:31.772797 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e03679-1ab5-48f4-8932-2db0759af0fd None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:31.773119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e03679-1ab5-48f4-8932-2db0759af0fd None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:31.773401 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e03679-1ab5-48f4-8932-2db0759af0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:31.795678 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-f6e03679-1ab5-48f4-8932-2db0759af0fd None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:31.796908 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f6e03679-1ab5-48f4-8932-2db0759af0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:31.824762 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:31.874669 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 146/288] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:31.913793 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a5a1b-d2f9-44be-a71a-d0f41b813865 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:32.031854 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a5a1b-d2f9-44be-a71a-d0f41b813865 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:32.041335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:32.068836 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286fa9d-f4b2-4b3e-8aa9-aa20ef64931c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:32.069509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286fa9d-f4b2-4b3e-8aa9-aa20ef64931c None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:32.069792 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286fa9d-f4b2-4b3e-8aa9-aa20ef64931c None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:32.070064 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286fa9d-f4b2-4b3e-8aa9-aa20ef64931c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:32.081666 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0286fa9d-f4b2-4b3e-8aa9-aa20ef64931c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:32.087558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-0286fa9d-f4b2-4b3e-8aa9-aa20ef64931c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:32.087558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:32.087558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:32.087558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:32.087558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:32.087558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:32.087558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:32.087558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:32.087558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:32.087558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:32.087558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:32.087558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:32.087558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:32.087558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:32.087558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:32.087558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:32.088547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:32.088547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:32.088547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:32.088547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:32.088547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:32.088547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:32.088547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:32.088547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:32.088547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:32.088547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:32.088547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:32.088547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:32.088547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:32.088547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:32.088547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:32.088547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:32.089344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:32.089344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:32.089344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:32.089344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:32.089344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:32.089344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:32.089344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:32.089344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:32.089344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:32.089344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:32.089344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:32.089344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:32.091961 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 143/289] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:32.097562 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e03679-1ab5-48f4-8932-2db0759af0fd None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:32.236683 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e03679-1ab5-48f4-8932-2db0759af0fd None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:32.283618 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558e17c-61ec-45a5-b5c2-fa4845da6cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:32.284170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558e17c-61ec-45a5-b5c2-fa4845da6cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:32.284752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558e17c-61ec-45a5-b5c2-fa4845da6cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:32.285074 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558e17c-61ec-45a5-b5c2-fa4845da6cc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:32.285786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558e17c-61ec-45a5-b5c2-fa4845da6cc9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:32.286092 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558e17c-61ec-45a5-b5c2-fa4845da6cc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:32.287459 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558e17c-61ec-45a5-b5c2-fa4845da6cc9 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:32.300765 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 147/290] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:32.302179 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286fa9d-f4b2-4b3e-8aa9-aa20ef64931c None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:32.421356 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286fa9d-f4b2-4b3e-8aa9-aa20ef64931c None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:32.455896 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19bf1b8-16c0-48f3-b18d-1d1917443a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:32.456934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19bf1b8-16c0-48f3-b18d-1d1917443a49 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:32.457410 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19bf1b8-16c0-48f3-b18d-1d1917443a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:32.457873 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19bf1b8-16c0-48f3-b18d-1d1917443a49 None admin] PATH_INFO: `/v3/users/19c767183b0d491491c66eb1d4a0b02f` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:32.459230 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19bf1b8-16c0-48f3-b18d-1d1917443a49 None admin] RBAC: Authorizing `identity:get_user(user_id=19c767183b0d491491c66eb1d4a0b02f)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:32.460956 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19bf1b8-16c0-48f3-b18d-1d1917443a49 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:32.462754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 144/291] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:32 2022] GET /identity/v3/users/19c767183b0d491491c66eb1d4a0b02f => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:32.473998 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558e17c-61ec-45a5-b5c2-fa4845da6cc9 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:32.511681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:32.605106 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558e17c-61ec-45a5-b5c2-fa4845da6cc9 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:32.647712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c64063-645f-4a76-ba10-856593e52f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:32.648346 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c64063-645f-4a76-ba10-856593e52f4d None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:32.648626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c64063-645f-4a76-ba10-856593e52f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:32.648929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c64063-645f-4a76-ba10-856593e52f4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:32.734340 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c64063-645f-4a76-ba10-856593e52f4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:32.735799 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c64063-645f-4a76-ba10-856593e52f4d None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:32.772103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:32.855345 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 148/292] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:32.861733 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19bf1b8-16c0-48f3-b18d-1d1917443a49 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:32.955752 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:33.087717 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19bf1b8-16c0-48f3-b18d-1d1917443a49 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:33.125778 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69610f4-9fac-46ad-aff3-f97b655f12b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:33.126345 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69610f4-9fac-46ad-aff3-f97b655f12b8 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:33.126630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69610f4-9fac-46ad-aff3-f97b655f12b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:33.126904 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69610f4-9fac-46ad-aff3-f97b655f12b8 None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:33.128070 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69610f4-9fac-46ad-aff3-f97b655f12b8 None admin] RBAC: Authorizing `identity:get_project(project_id=6a66a60a14e247e5bf2142da6f43cc44)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:33.129566 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69610f4-9fac-46ad-aff3-f97b655f12b8 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:33.131536 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 145/293] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:32 2022] GET /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44 => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:33.137063 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c64063-645f-4a76-ba10-856593e52f4d None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:33.263053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fe032-421c-4c7c-a6a8-1e8482ac61be None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:33.263515 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fe032-421c-4c7c-a6a8-1e8482ac61be None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:33.263893 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fe032-421c-4c7c-a6a8-1e8482ac61be None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:33.265535 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 146/294] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:22:33.270338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf7eb6-ade7-48d3-9778-967a2e52eb00 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:33.270692 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf7eb6-ade7-48d3-9778-967a2e52eb00 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:33.271044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf7eb6-ade7-48d3-9778-967a2e52eb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:33.289107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-dcdf7eb6-ade7-48d3-9778-967a2e52eb00 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:33.290436 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-dcdf7eb6-ade7-48d3-9778-967a2e52eb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:33.306695 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c64063-645f-4a76-ba10-856593e52f4d None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:33.335055 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:33.347645 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98d626-a9be-4bf8-97c1-9ef8292949f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:33.348270 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98d626-a9be-4bf8-97c1-9ef8292949f9 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:33.348552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98d626-a9be-4bf8-97c1-9ef8292949f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:33.348828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98d626-a9be-4bf8-97c1-9ef8292949f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:33.349641 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad98d626-a9be-4bf8-97c1-9ef8292949f9 None admin] RBAC: Adding query filter params (role.id=8de48592893c43789e1d40158943fd52, scope.project.id=6a66a60a14e247e5bf2142da6f43cc44, user.id=19c767183b0d491491c66eb1d4a0b02f) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:33.349958 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad98d626-a9be-4bf8-97c1-9ef8292949f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:33.351396 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad98d626-a9be-4bf8-97c1-9ef8292949f9 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:33.357573 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 149/295] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:22:33 2022] GET /identity/v3/role_assignments?user.id=19c767183b0d491491c66eb1d4a0b02f&scope.project.id=6a66a60a14e247e5bf2142da6f43cc44&role.id=8de48592893c43789e1d40158943fd52 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:33.358757 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a88bfb-4a0b-45e8-ace7-60d088a59354 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:33.359043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a88bfb-4a0b-45e8-ace7-60d088a59354 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:33.359322 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a88bfb-4a0b-45e8-ace7-60d088a59354 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:33.360698 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 150/296] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:33.361684 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128102ab-34b8-4f59-8a91-fee19cfe98b3 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:33.361977 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128102ab-34b8-4f59-8a91-fee19cfe98b3 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:33.362259 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128102ab-34b8-4f59-8a91-fee19cfe98b3 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:33.363742 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 151/297] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:33.377119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1021def-36a0-4ea9-b18a-27b57855477e None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:33.377119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1021def-36a0-4ea9-b18a-27b57855477e None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:33.377119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1021def-36a0-4ea9-b18a-27b57855477e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:33.393083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 147/298] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:33.394715 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4f529-a63e-4d26-9930-47fd4c9777d5 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:33.394990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-c1021def-36a0-4ea9-b18a-27b57855477e None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:33.395108 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4f529-a63e-4d26-9930-47fd4c9777d5 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:33.395259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4f529-a63e-4d26-9930-47fd4c9777d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:33.396220 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-c1021def-36a0-4ea9-b18a-27b57855477e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:33.430994 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-9fa4f529-a63e-4d26-9930-47fd4c9777d5 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:33.432808 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9fa4f529-a63e-4d26-9930-47fd4c9777d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:33.473298 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:33.477073 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 152/299] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:33.478048 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1021def-36a0-4ea9-b18a-27b57855477e None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:33.530853 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 148/300] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:33.531725 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4f529-a63e-4d26-9930-47fd4c9777d5 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:33.600567 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1021def-36a0-4ea9-b18a-27b57855477e None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:33.618157 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:33.640098 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c34ca-8cf3-446e-9ba8-4b5f9a283dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:33.640786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c34ca-8cf3-446e-9ba8-4b5f9a283dfc None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:33.641158 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c34ca-8cf3-446e-9ba8-4b5f9a283dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:33.641503 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c34ca-8cf3-446e-9ba8-4b5f9a283dfc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:33.653167 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c34ca-8cf3-446e-9ba8-4b5f9a283dfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:33.658576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-3d7c34ca-8cf3-446e-9ba8-4b5f9a283dfc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 16:22:33.658576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:33.658576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:33.658576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:33.658576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:33.658576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:33.658576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:33.658576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:33.658576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:33.658576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:33.658576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:33.658576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:33.658576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:33.658576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:33.658576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:33.658576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:33.659447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:33.659447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:33.659447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:33.659447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:33.659447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:33.659447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:33.659447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:33.659447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:33.659447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:33.659447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:33.659447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:33.659447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:33.659447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:33.659447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:33.659447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:33.659447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:33.660236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:33.660236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:33.660236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:33.660236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:33.660236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:33.660236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:33.660236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:33.660236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:33.660236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:33.660236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:33.660236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 16:22:33.660236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:33.660236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 153/301] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:33.660979 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c34ca-8cf3-446e-9ba8-4b5f9a283dfc None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:33.667354 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4f529-a63e-4d26-9930-47fd4c9777d5 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:33.703640 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55961f-fdb5-4ad3-9a31-424ee8efaa48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:33.704259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55961f-fdb5-4ad3-9a31-424ee8efaa48 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:33.704537 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55961f-fdb5-4ad3-9a31-424ee8efaa48 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:33.704810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55961f-fdb5-4ad3-9a31-424ee8efaa48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:33.715091 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c55961f-fdb5-4ad3-9a31-424ee8efaa48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:33.720036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-9c55961f-fdb5-4ad3-9a31-424ee8efaa48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:33.720036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:33.720036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:33.720036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:33.720036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:33.720036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:33.720036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:33.720036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:33.720036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:33.720036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:33.720036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:33.720036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:33.720036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:33.720036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:33.720036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:33.720036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:33.720933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:33.720933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:33.720933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:33.720933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:33.720933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:33.720933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:33.720933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:33.720933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:33.720933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:33.720933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:33.720933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:33.720933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:33.720933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:33.720933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:33.720933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:33.720933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:33.721734 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:33.721734 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:33.721734 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:33.721734 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:33.721734 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:33.721734 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:33.721734 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:33.721734 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:33.721734 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:33.721734 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:33.721734 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:33.721734 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:33.721734 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 149/302] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:33.722902 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55961f-fdb5-4ad3-9a31-424ee8efaa48 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:33.749637 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:33.776528 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c34ca-8cf3-446e-9ba8-4b5f9a283dfc None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:33.810256 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608b5b3-46a9-49d8-bcf9-0f000762f56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:33.810891 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608b5b3-46a9-49d8-bcf9-0f000762f56c None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:33.811171 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608b5b3-46a9-49d8-bcf9-0f000762f56c None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:33.811441 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608b5b3-46a9-49d8-bcf9-0f000762f56c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:33.812584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608b5b3-46a9-49d8-bcf9-0f000762f56c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:33.814281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608b5b3-46a9-49d8-bcf9-0f000762f56c None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:33.820039 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 154/303] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:22:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:33.820039 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608b5b3-46a9-49d8-bcf9-0f000762f56c None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:33.833431 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55961f-fdb5-4ad3-9a31-424ee8efaa48 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:33.870107 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:33.873029 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4e133-654f-49b7-88b0-fb7101ce7a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:33.873666 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4e133-654f-49b7-88b0-fb7101ce7a99 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:33.874032 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4e133-654f-49b7-88b0-fb7101ce7a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:33.874221 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4e133-654f-49b7-88b0-fb7101ce7a99 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:33.874947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4e133-654f-49b7-88b0-fb7101ce7a99 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:33.875253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4e133-654f-49b7-88b0-fb7101ce7a99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:33.876649 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4e133-654f-49b7-88b0-fb7101ce7a99 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:33.885219 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 150/304] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:33.886005 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4e133-654f-49b7-88b0-fb7101ce7a99 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:33.928079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608b5b3-46a9-49d8-bcf9-0f000762f56c None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:33.964558 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3dbc7-3321-40a2-89ff-9d7255ea6ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:33.965348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3dbc7-3321-40a2-89ff-9d7255ea6ccd None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:33.965660 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3dbc7-3321-40a2-89ff-9d7255ea6ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:33.965990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3dbc7-3321-40a2-89ff-9d7255ea6ccd None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:33.966824 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3dbc7-3321-40a2-89ff-9d7255ea6ccd None admin] RBAC: Adding query filter params (name=member) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:33.967689 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3dbc7-3321-40a2-89ff-9d7255ea6ccd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:33.968651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3dbc7-3321-40a2-89ff-9d7255ea6ccd None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:33.981356 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 155/305] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:33.981356 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3dbc7-3321-40a2-89ff-9d7255ea6ccd None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:33.986708 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:34.002692 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4e133-654f-49b7-88b0-fb7101ce7a99 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:34.036313 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d185944-3138-4561-92c0-ce7b52baf5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:34.036962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d185944-3138-4561-92c0-ce7b52baf5f7 None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:34.037247 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d185944-3138-4561-92c0-ce7b52baf5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:34.037576 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d185944-3138-4561-92c0-ce7b52baf5f7 None admin] PATH_INFO: `/v3/groups` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:34.038233 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d185944-3138-4561-92c0-ce7b52baf5f7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:34.039735 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d185944-3138-4561-92c0-ce7b52baf5f7 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:34.054967 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 151/306] 10.222.0.74 () {60 vars in 1134 bytes} [Wed Jul 27 16:22:33 2022] POST /identity/v3/groups => generated 229 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:22:34.055753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d185944-3138-4561-92c0-ce7b52baf5f7 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:34.098981 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3dbc7-3321-40a2-89ff-9d7255ea6ccd None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:34.110950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:34.142809 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7db8e-10aa-4aab-9d07-af6f8c0a76bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:34.144184 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7db8e-10aa-4aab-9d07-af6f8c0a76bd None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:34.144464 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7db8e-10aa-4aab-9d07-af6f8c0a76bd None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:34.144736 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7db8e-10aa-4aab-9d07-af6f8c0a76bd None admin] PATH_INFO: `/v3/users/658f5cb042a1480b9a1fc58d44ce78bc` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:34.163050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7db8e-10aa-4aab-9d07-af6f8c0a76bd None admin] RBAC: Authorizing `identity:get_user(user_id=658f5cb042a1480b9a1fc58d44ce78bc)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:34.164807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7db8e-10aa-4aab-9d07-af6f8c0a76bd None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:34.169147 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 156/307] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:33 2022] GET /identity/v3/users/658f5cb042a1480b9a1fc58d44ce78bc => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:34.169147 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e1bc4-8610-4e51-a374-114e5a4d0c87 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:34.169147 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e1bc4-8610-4e51-a374-114e5a4d0c87 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:34.169540 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e1bc4-8610-4e51-a374-114e5a4d0c87 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:34.173123 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 157/308] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:34.173283 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e1bc4-8610-4e51-a374-114e5a4d0c87 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:34.179190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d185944-3138-4561-92c0-ce7b52baf5f7 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:34.219711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511b8a7-1ac9-453f-a748-a923fe08966b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:34.220371 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511b8a7-1ac9-453f-a748-a923fe08966b None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:34.220833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511b8a7-1ac9-453f-a748-a923fe08966b None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:34.221258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511b8a7-1ac9-453f-a748-a923fe08966b None admin] PATH_INFO: `/v3/users/d3f366d1181041a1a4fe50d790382201` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:34.222543 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511b8a7-1ac9-453f-a748-a923fe08966b None admin] RBAC: Authorizing `identity:get_user(user_id=d3f366d1181041a1a4fe50d790382201)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:34.224771 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511b8a7-1ac9-453f-a748-a923fe08966b None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:34.226442 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 152/309] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:34 2022] GET /identity/v3/users/d3f366d1181041a1a4fe50d790382201 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:34.227611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd10df-15f4-4744-a41c-e70d817b29c6 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:34.227898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd10df-15f4-4744-a41c-e70d817b29c6 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:34.228168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd10df-15f4-4744-a41c-e70d817b29c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:34.249181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-59cd10df-15f4-4744-a41c-e70d817b29c6 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:34.250391 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-59cd10df-15f4-4744-a41c-e70d817b29c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:34.260783 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:34.296620 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e1bc4-8610-4e51-a374-114e5a4d0c87 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:34.325515 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 153/310] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:34.326634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd10df-15f4-4744-a41c-e70d817b29c6 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:34.338314 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0c39f-c384-4409-8bb0-787c059d8354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:34.338944 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0c39f-c384-4409-8bb0-787c059d8354 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:34.339225 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0c39f-c384-4409-8bb0-787c059d8354 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:34.339497 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0c39f-c384-4409-8bb0-787c059d8354 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:34.340293 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0c39f-c384-4409-8bb0-787c059d8354 None admin] RBAC: Adding query filter params (role.id=1204be9fc6d7483fb28ba04dfa5b3f6e, user.id=658f5cb042a1480b9a1fc58d44ce78bc, scope.system=all) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:34.341187 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0c39f-c384-4409-8bb0-787c059d8354 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:34.342804 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0c39f-c384-4409-8bb0-787c059d8354 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:34.349004 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 158/311] 10.222.0.74 () {56 vars in 1308 bytes} [Wed Jul 27 16:22:34 2022] GET /identity/v3/role_assignments?user.id=658f5cb042a1480b9a1fc58d44ce78bc&scope.system=all&role.id=1204be9fc6d7483fb28ba04dfa5b3f6e => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:34.352444 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0c39f-c384-4409-8bb0-787c059d8354 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:34.398588 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:34.469477 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd10df-15f4-4744-a41c-e70d817b29c6 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:34.481798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0c39f-c384-4409-8bb0-787c059d8354 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:34.518060 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a95ff-88ae-462f-ab35-edd80d63d369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:34.519112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a95ff-88ae-462f-ab35-edd80d63d369 None admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:34.519591 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a95ff-88ae-462f-ab35-edd80d63d369 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:34.520045 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a95ff-88ae-462f-ab35-edd80d63d369 None admin] PATH_INFO: `/v3/system/users/d3f366d1181041a1a4fe50d790382201/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:34.521809 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a4eee-0117-46be-ba67-8d074e0b9d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:34.522066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991a95ff-88ae-462f-ab35-edd80d63d369 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d3f366d1181041a1a4fe50d790382201, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:34.522580 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a4eee-0117-46be-ba67-8d074e0b9d3b None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:34.522937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a4eee-0117-46be-ba67-8d074e0b9d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:34.523275 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a4eee-0117-46be-ba67-8d074e0b9d3b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:34.524195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991a95ff-88ae-462f-ab35-edd80d63d369 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:34.544929 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 154/312] 10.222.0.74 () {58 vars in 1334 bytes} [Wed Jul 27 16:22:34 2022] PUT /identity/v3/system/users/d3f366d1181041a1a4fe50d790382201/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:22:34.546489 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a4eee-0117-46be-ba67-8d074e0b9d3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:34.552192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-380a4eee-0117-46be-ba67-8d074e0b9d3b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 16:22:34.552192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:34.552192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:34.552192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:34.552192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:34.552192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:34.552192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:34.552192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:34.552192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:34.552192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:34.552192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:34.552192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:34.552192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:34.552192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:34.552192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:34.552192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:34.553145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:34.553145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:34.553145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:34.553145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:34.553145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:34.553145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:34.553145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:34.553145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:34.553145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:34.553145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:34.553145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:34.553145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:34.553145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:34.553145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:34.553145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:34.553145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:34.554165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:34.554165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:34.554165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:34.554165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:34.554165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:34.554165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:34.554165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:34.554165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:34.554165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:34.554165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:34.554165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 16:22:34.554165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:34.554165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 159/313] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:22:34.560247 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a95ff-88ae-462f-ab35-edd80d63d369 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:34.573329 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:34.674411 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a95ff-88ae-462f-ab35-edd80d63d369 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:34.711518 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7dd64-f679-41bd-8d8e-22ec7eda5fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:34.712136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7dd64-f679-41bd-8d8e-22ec7eda5fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:34.712413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7dd64-f679-41bd-8d8e-22ec7eda5fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:34.712681 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7dd64-f679-41bd-8d8e-22ec7eda5fa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:34.713425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7dd64-f679-41bd-8d8e-22ec7eda5fa5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:34.713728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7dd64-f679-41bd-8d8e-22ec7eda5fa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:34.715110 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7dd64-f679-41bd-8d8e-22ec7eda5fa5 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:34.723933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 155/314] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:34.729444 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a4eee-0117-46be-ba67-8d074e0b9d3b None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:34.828776 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:34.839585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a4eee-0117-46be-ba67-8d074e0b9d3b None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:34.877470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fd3fc-98c5-43c6-b5ec-958683bd9250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:34.878278 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fd3fc-98c5-43c6-b5ec-958683bd9250 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:34.878651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fd3fc-98c5-43c6-b5ec-958683bd9250 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:34.878937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fd3fc-98c5-43c6-b5ec-958683bd9250 None admin] PATH_INFO: `/v3/users/26eed0d8b1874a2e8b8175d992043d06` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:34.880160 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fd3fc-98c5-43c6-b5ec-958683bd9250 None admin] RBAC: Authorizing `identity:get_user(user_id=26eed0d8b1874a2e8b8175d992043d06)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:34.882050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fd3fc-98c5-43c6-b5ec-958683bd9250 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:34.884195 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 160/315] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:34 2022] GET /identity/v3/users/26eed0d8b1874a2e8b8175d992043d06 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:34.889590 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7dd64-f679-41bd-8d8e-22ec7eda5fa5 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:34.966498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:35.000670 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7dd64-f679-41bd-8d8e-22ec7eda5fa5 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:35.037507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03426ea-8ebc-41b2-b5c5-c322214ebd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:35.038135 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03426ea-8ebc-41b2-b5c5-c322214ebd32 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:35.038421 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03426ea-8ebc-41b2-b5c5-c322214ebd32 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:35.038702 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03426ea-8ebc-41b2-b5c5-c322214ebd32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:35.112479 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03426ea-8ebc-41b2-b5c5-c322214ebd32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:35.113999 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03426ea-8ebc-41b2-b5c5-c322214ebd32 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:35.185681 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:35.222731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 156/316] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:35.228224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fd3fc-98c5-43c6-b5ec-958683bd9250 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:35.332230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fd3fc-98c5-43c6-b5ec-958683bd9250 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:35.368009 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ee502-bd62-4166-b4b4-f8f6f558c08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:35.369058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ee502-bd62-4166-b4b4-f8f6f558c08f None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:35.369588 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ee502-bd62-4166-b4b4-f8f6f558c08f None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:35.370074 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ee502-bd62-4166-b4b4-f8f6f558c08f None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:35.371442 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ee502-bd62-4166-b4b4-f8f6f558c08f None admin] RBAC: Authorizing `identity:get_project(project_id=6a66a60a14e247e5bf2142da6f43cc44)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:35.373648 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ee502-bd62-4166-b4b4-f8f6f558c08f None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:35.375821 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 161/317] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:35 2022] GET /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:35.381530 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03426ea-8ebc-41b2-b5c5-c322214ebd32 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:35.482080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03426ea-8ebc-41b2-b5c5-c322214ebd32 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:35.518829 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5652a66-bba6-410d-9e89-19f5b621c8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:35.519830 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5652a66-bba6-410d-9e89-19f5b621c8be None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:35.520372 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5652a66-bba6-410d-9e89-19f5b621c8be None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:35.520843 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5652a66-bba6-410d-9e89-19f5b621c8be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:35.521980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5652a66-bba6-410d-9e89-19f5b621c8be None admin] RBAC: Adding query filter params (role.id=1204be9fc6d7483fb28ba04dfa5b3f6e, scope.project.id=6a66a60a14e247e5bf2142da6f43cc44, user.id=26eed0d8b1874a2e8b8175d992043d06) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:35.522534 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5652a66-bba6-410d-9e89-19f5b621c8be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:35.524271 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5652a66-bba6-410d-9e89-19f5b621c8be None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:35.531853 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 157/318] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:22:35 2022] GET /identity/v3/role_assignments?user.id=26eed0d8b1874a2e8b8175d992043d06&scope.project.id=6a66a60a14e247e5bf2142da6f43cc44&role.id=1204be9fc6d7483fb28ba04dfa5b3f6e => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:36.148183 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd6beb-27dc-4093-b5dd-8c14f79d2ce0 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:36.148561 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd6beb-27dc-4093-b5dd-8c14f79d2ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:36.148756 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd6beb-27dc-4093-b5dd-8c14f79d2ce0 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:36.150180 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 162/319] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:36.155290 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b00916-c5af-44a5-b963-10fc2c56eb3a None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:36.155594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b00916-c5af-44a5-b963-10fc2c56eb3a None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:36.155876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b00916-c5af-44a5-b963-10fc2c56eb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:36.176341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-e4b00916-c5af-44a5-b963-10fc2c56eb3a None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:36.177578 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-e4b00916-c5af-44a5-b963-10fc2c56eb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:36.208878 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:36.252815 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 158/320] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:36.274925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd6beb-27dc-4093-b5dd-8c14f79d2ce0 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:36.303632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:36.379781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd6beb-27dc-4093-b5dd-8c14f79d2ce0 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:36.414212 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40b067-508e-40a7-842c-ab016ecf0a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:36.414776 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40b067-508e-40a7-842c-ab016ecf0a7f None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:36.415057 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40b067-508e-40a7-842c-ab016ecf0a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:36.415332 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40b067-508e-40a7-842c-ab016ecf0a7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:36.416472 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40b067-508e-40a7-842c-ab016ecf0a7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:36.417966 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40b067-508e-40a7-842c-ab016ecf0a7f None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:36.420226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 163/321] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:22:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:36.425671 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b00916-c5af-44a5-b963-10fc2c56eb3a None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:36.425927 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f2574-6da3-418c-a3da-268f81332014 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:36.426208 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f2574-6da3-418c-a3da-268f81332014 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:36.426479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f2574-6da3-418c-a3da-268f81332014 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:36.427884 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 164/322] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:36.432782 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9828afd-2b2c-498e-b26f-5888b7126f6d None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:36.433100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9828afd-2b2c-498e-b26f-5888b7126f6d None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:36.433378 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9828afd-2b2c-498e-b26f-5888b7126f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:36.451843 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-e9828afd-2b2c-498e-b26f-5888b7126f6d None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:36.453066 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e9828afd-2b2c-498e-b26f-5888b7126f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:36.522176 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 165/323] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:36.543412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9828afd-2b2c-498e-b26f-5888b7126f6d None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:36.545003 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b00916-c5af-44a5-b963-10fc2c56eb3a None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:36.568829 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:36.583432 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71823e4-ef3d-4dc9-b876-159694c5a30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:36.584044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71823e4-ef3d-4dc9-b876-159694c5a30b None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:36.584325 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71823e4-ef3d-4dc9-b876-159694c5a30b None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:36.584595 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71823e4-ef3d-4dc9-b876-159694c5a30b None admin] PATH_INFO: `/v3/groups` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:36.585284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71823e4-ef3d-4dc9-b876-159694c5a30b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:36.586720 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71823e4-ef3d-4dc9-b876-159694c5a30b None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:36.601158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 159/324] 10.222.0.74 () {60 vars in 1134 bytes} [Wed Jul 27 16:22:36 2022] POST /identity/v3/groups => generated 226 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:22:36.602344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c28f9-2576-4b0d-bdfa-bcb73cda81f4 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:36.602630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c28f9-2576-4b0d-bdfa-bcb73cda81f4 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:36.602901 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c28f9-2576-4b0d-bdfa-bcb73cda81f4 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:36.604263 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 160/325] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:36.610190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc71cf-7bb3-4f14-b1ac-9758c6f79a71 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:36.610474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc71cf-7bb3-4f14-b1ac-9758c6f79a71 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:36.610753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc71cf-7bb3-4f14-b1ac-9758c6f79a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:36.630164 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-31cc71cf-7bb3-4f14-b1ac-9758c6f79a71 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:36.631398 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-31cc71cf-7bb3-4f14-b1ac-9758c6f79a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:36.651512 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:36.667428 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9828afd-2b2c-498e-b26f-5888b7126f6d None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:36.706013 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae98bf-36e8-49c8-9b81-355c60e1d4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:36.706673 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae98bf-36e8-49c8-9b81-355c60e1d4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:36.706951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae98bf-36e8-49c8-9b81-355c60e1d4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:36.707218 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae98bf-36e8-49c8-9b81-355c60e1d4c5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:36.708867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 161/326] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:36.718965 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae98bf-36e8-49c8-9b81-355c60e1d4c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:36.724079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-e6ae98bf-36e8-49c8-9b81-355c60e1d4c5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 16:22:36.724079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:36.724079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:36.724079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:36.724079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:36.724079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:36.724079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:36.724079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:36.724079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:36.724079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:36.724079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:36.724079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:36.724079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:36.724079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:36.724079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:36.724079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:36.725035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:36.725035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:36.725035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:36.725035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:36.725035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:36.725035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:36.725035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:36.725035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:36.725035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:36.725035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:36.725035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:36.725035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:36.725035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:36.725035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:36.725035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:36.725035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:36.725828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:36.725828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:36.725828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:36.725828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:36.725828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:36.725828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:36.725828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:36.725828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:36.725828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:36.725828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:36.725828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 16:22:36.725828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:36.725828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 166/327] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:22:36.733445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae98bf-36e8-49c8-9b81-355c60e1d4c5 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:36.733639 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc71cf-7bb3-4f14-b1ac-9758c6f79a71 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:36.827816 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:36.846487 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae98bf-36e8-49c8-9b81-355c60e1d4c5 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:36.859617 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc71cf-7bb3-4f14-b1ac-9758c6f79a71 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:36.898244 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481cd3a8-407d-4473-a3c2-fafe8ba797dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:36.899208 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cd3a8-407d-4473-a3c2-fafe8ba797dc None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:36.899700 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cd3a8-407d-4473-a3c2-fafe8ba797dc None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:36.900154 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cd3a8-407d-4473-a3c2-fafe8ba797dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:36.907301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43a065-d734-4020-a49a-1e901ec90f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:36.907929 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43a065-d734-4020-a49a-1e901ec90f33 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:36.908199 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43a065-d734-4020-a49a-1e901ec90f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:36.908459 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43a065-d734-4020-a49a-1e901ec90f33 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:36.909229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43a065-d734-4020-a49a-1e901ec90f33 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:36.909527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43a065-d734-4020-a49a-1e901ec90f33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:36.910912 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43a065-d734-4020-a49a-1e901ec90f33 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:36.914048 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481cd3a8-407d-4473-a3c2-fafe8ba797dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:36.919541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-481cd3a8-407d-4473-a3c2-fafe8ba797dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:36.919541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:36.919541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:36.919541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:36.919541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:36.919541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:36.919541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:36.919541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:36.919541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:36.919541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:36.919541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:36.919541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:36.919541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:36.919541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:36.919541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:36.919541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:36.920402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:36.920402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:36.920402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:36.920402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:36.920402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:36.920402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:36.920402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:36.920402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:36.920402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:36.920402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:36.920402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:36.920402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:36.920402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:36.920402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:36.920402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:36.920402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:36.921224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:36.921224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:36.921224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:36.921224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:36.921224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:36.921224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:36.921224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:36.921224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:36.921224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:36.921224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:36.921224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:36.921224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:36.921820 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 162/328] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:36.923240 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 167/329] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:36.925344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43a065-d734-4020-a49a-1e901ec90f33 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:36.929518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481cd3a8-407d-4473-a3c2-fafe8ba797dc None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:36.946546 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:37.049725 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43a065-d734-4020-a49a-1e901ec90f33 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:37.055015 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481cd3a8-407d-4473-a3c2-fafe8ba797dc None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:37.100275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb2ac7-d771-43c0-8f67-0bf199f2e3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:37.100956 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb2ac7-d771-43c0-8f67-0bf199f2e3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:37.101286 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb2ac7-d771-43c0-8f67-0bf199f2e3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:37.101567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb2ac7-d771-43c0-8f67-0bf199f2e3d1 None admin] PATH_INFO: `/v3/users/658f5cb042a1480b9a1fc58d44ce78bc` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:37.103351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb2ac7-d771-43c0-8f67-0bf199f2e3d1 None admin] RBAC: Authorizing `identity:get_user(user_id=658f5cb042a1480b9a1fc58d44ce78bc)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:37.103858 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d81a3-b9c2-46ec-a68a-d65d251f3595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:37.104488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d81a3-b9c2-46ec-a68a-d65d251f3595 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:37.104768 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d81a3-b9c2-46ec-a68a-d65d251f3595 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:37.104876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb2ac7-d771-43c0-8f67-0bf199f2e3d1 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:37.105066 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d81a3-b9c2-46ec-a68a-d65d251f3595 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:37.105796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d81a3-b9c2-46ec-a68a-d65d251f3595 None admin] RBAC: Adding query filter params (name=member) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:37.106658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d81a3-b9c2-46ec-a68a-d65d251f3595 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:37.107412 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 163/330] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:36 2022] GET /identity/v3/users/658f5cb042a1480b9a1fc58d44ce78bc => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:37.109911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d81a3-b9c2-46ec-a68a-d65d251f3595 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:37.113961 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb2ac7-d771-43c0-8f67-0bf199f2e3d1 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:37.127812 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 168/331] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:37.132541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d81a3-b9c2-46ec-a68a-d65d251f3595 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:37.149259 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:37.229205 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:37.247497 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb2ac7-d771-43c0-8f67-0bf199f2e3d1 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:37.285688 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d702b3-7193-4e6b-a54a-d3e8f5cec776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:37.286396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d702b3-7193-4e6b-a54a-d3e8f5cec776 None admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:37.286675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d702b3-7193-4e6b-a54a-d3e8f5cec776 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:37.286951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d702b3-7193-4e6b-a54a-d3e8f5cec776 None admin] PATH_INFO: `/v3/system/users/658f5cb042a1480b9a1fc58d44ce78bc/roles/1204be9fc6d7483fb28ba04dfa5b3f6e` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:37.290798 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d702b3-7193-4e6b-a54a-d3e8f5cec776 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=658f5cb042a1480b9a1fc58d44ce78bc, role_id=1204be9fc6d7483fb28ba04dfa5b3f6e)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:37.292249 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d702b3-7193-4e6b-a54a-d3e8f5cec776 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:37.306269 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 164/332] 10.222.0.74 () {58 vars in 1334 bytes} [Wed Jul 27 16:22:37 2022] PUT /identity/v3/system/users/658f5cb042a1480b9a1fc58d44ce78bc/roles/1204be9fc6d7483fb28ba04dfa5b3f6e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:22:37.327348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d81a3-b9c2-46ec-a68a-d65d251f3595 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:37.368470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f32ddf-64fc-4b52-a5b4-4e914689ca58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:37.369237 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f32ddf-64fc-4b52-a5b4-4e914689ca58 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:37.369515 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f32ddf-64fc-4b52-a5b4-4e914689ca58 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:37.369784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f32ddf-64fc-4b52-a5b4-4e914689ca58 None admin] PATH_INFO: `/v3/users/d3f366d1181041a1a4fe50d790382201` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:37.371035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f32ddf-64fc-4b52-a5b4-4e914689ca58 None admin] RBAC: Authorizing `identity:get_user(user_id=d3f366d1181041a1a4fe50d790382201)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:37.372656 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f32ddf-64fc-4b52-a5b4-4e914689ca58 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:37.374732 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 169/333] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:37 2022] GET /identity/v3/users/d3f366d1181041a1a4fe50d790382201 => generated 293 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:37.380639 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d702b3-7193-4e6b-a54a-d3e8f5cec776 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:37.532750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d702b3-7193-4e6b-a54a-d3e8f5cec776 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:37.571496 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73b51c-ecaf-4de7-bf38-3c2adc5a18fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:37.572127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73b51c-ecaf-4de7-bf38-3c2adc5a18fc None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:37.572409 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73b51c-ecaf-4de7-bf38-3c2adc5a18fc None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:37.572685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73b51c-ecaf-4de7-bf38-3c2adc5a18fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:37.573516 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73b51c-ecaf-4de7-bf38-3c2adc5a18fc None admin] RBAC: Adding query filter params (role.id=8de48592893c43789e1d40158943fd52, user.id=d3f366d1181041a1a4fe50d790382201, scope.system=all) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:37.573824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73b51c-ecaf-4de7-bf38-3c2adc5a18fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:37.575629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73b51c-ecaf-4de7-bf38-3c2adc5a18fc None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:37.588638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 165/334] 10.222.0.74 () {56 vars in 1308 bytes} [Wed Jul 27 16:22:37 2022] GET /identity/v3/role_assignments?user.id=d3f366d1181041a1a4fe50d790382201&scope.system=all&role.id=8de48592893c43789e1d40158943fd52 => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:38.924378 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f2e06-e73c-40ad-9919-33fc6b3a22e4 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:38.924913 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f2e06-e73c-40ad-9919-33fc6b3a22e4 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:38.924993 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f2e06-e73c-40ad-9919-33fc6b3a22e4 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:38.926552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 170/335] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:38.933077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c15f0-4b2c-4763-aa7f-ec58506e8bf7 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:38.933518 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c15f0-4b2c-4763-aa7f-ec58506e8bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:38.933689 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c15f0-4b2c-4763-aa7f-ec58506e8bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:38.948277 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:38.955147 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-0c0c15f0-4b2c-4763-aa7f-ec58506e8bf7 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:38.962403 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0c0c15f0-4b2c-4763-aa7f-ec58506e8bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:39.044795 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 166/336] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:39.080428 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f2e06-e73c-40ad-9919-33fc6b3a22e4 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:39.191982 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:39.225836 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f2e06-e73c-40ad-9919-33fc6b3a22e4 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:39.271762 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caade53a-9531-4d37-af78-b7de709bb0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:39.272418 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caade53a-9531-4d37-af78-b7de709bb0fa None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:39.272698 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caade53a-9531-4d37-af78-b7de709bb0fa None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:39.272998 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caade53a-9531-4d37-af78-b7de709bb0fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:39.288485 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caade53a-9531-4d37-af78-b7de709bb0fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:39.295046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-caade53a-9531-4d37-af78-b7de709bb0fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:39.295046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:39.295046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:39.295046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:39.295046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:39.295046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:39.295046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:39.295046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:39.295046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:39.295046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:39.295046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:39.295046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:39.295046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:39.295046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:39.295046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:39.295046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:39.296703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:39.296703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:39.296703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:39.296703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:39.296703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:39.296703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:39.296703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:39.296703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:39.296703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:39.296703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:39.296703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:39.296703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:39.296703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:39.296703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:39.296703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:39.296703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:39.298200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:39.298200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:39.298200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:39.298200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:39.298200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:39.298200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:39.298200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:39.298200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:39.298200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:39.298200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:39.298200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:39.298200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:39.298200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 171/337] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:39.306454 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c15f0-4b2c-4763-aa7f-ec58506e8bf7 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:39.377021 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe7698-11b6-4ef3-b3ab-c2cbc7e6195f None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:39.377324 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe7698-11b6-4ef3-b3ab-c2cbc7e6195f None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:39.377566 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe7698-11b6-4ef3-b3ab-c2cbc7e6195f None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:39.379151 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 172/338] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:39.385092 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61815ae6-f1ea-4ba0-a764-28fa1caca3f6 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:39.385761 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61815ae6-f1ea-4ba0-a764-28fa1caca3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:39.386506 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61815ae6-f1ea-4ba0-a764-28fa1caca3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:39.411193 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-61815ae6-f1ea-4ba0-a764-28fa1caca3f6 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:39.412825 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-61815ae6-f1ea-4ba0-a764-28fa1caca3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:39.437663 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c15f0-4b2c-4763-aa7f-ec58506e8bf7 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:39.474027 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94003c2-7ddb-4991-aa3d-c4aaa311b204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:39.474654 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94003c2-7ddb-4991-aa3d-c4aaa311b204 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:39.474934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94003c2-7ddb-4991-aa3d-c4aaa311b204 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:39.475208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94003c2-7ddb-4991-aa3d-c4aaa311b204 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:39.475925 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94003c2-7ddb-4991-aa3d-c4aaa311b204 None admin] RBAC: Adding query filter params (name=member) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:39.476385 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94003c2-7ddb-4991-aa3d-c4aaa311b204 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:39.477955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94003c2-7ddb-4991-aa3d-c4aaa311b204 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:39.480482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:39.489399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 167/339] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:39.491460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 173/340] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:39.495041 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94003c2-7ddb-4991-aa3d-c4aaa311b204 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:39.514885 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61815ae6-f1ea-4ba0-a764-28fa1caca3f6 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:39.587312 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:39.605130 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94003c2-7ddb-4991-aa3d-c4aaa311b204 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:39.632555 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61815ae6-f1ea-4ba0-a764-28fa1caca3f6 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:39.643069 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81233893-f3e7-454a-8deb-425db7d5e5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:39.643949 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81233893-f3e7-454a-8deb-425db7d5e5b0 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:39.644374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81233893-f3e7-454a-8deb-425db7d5e5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:39.645608 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81233893-f3e7-454a-8deb-425db7d5e5b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:39.676655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c982a-62f9-4734-a526-32ade5e1fac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:39.677586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c982a-62f9-4734-a526-32ade5e1fac8 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:39.678007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c982a-62f9-4734-a526-32ade5e1fac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:39.678422 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c982a-62f9-4734-a526-32ade5e1fac8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:39.690131 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c982a-62f9-4734-a526-32ade5e1fac8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:39.695328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-6a3c982a-62f9-4734-a526-32ade5e1fac8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:39.695328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:39.695328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:39.695328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:39.695328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:39.695328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:39.695328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:39.695328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:39.695328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:39.695328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:39.695328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:39.695328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:39.695328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:39.695328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:39.695328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:39.695328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:39.696188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:39.696188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:39.696188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:39.696188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:39.696188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:39.696188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:39.696188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:39.696188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:39.696188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:39.696188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:39.696188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:39.696188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:39.696188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:39.696188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:39.696188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:39.696188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:39.697239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:39.697239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:39.697239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:39.697239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:39.697239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:39.697239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:39.697239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:39.697239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:39.697239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:39.697239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:39.697239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:39.697239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:39.699259 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 174/341] 10.222.0.74 () {56 vars in 1114 bytes} [Wed Jul 27 16:22:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:22:39.705343 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c982a-62f9-4734-a526-32ade5e1fac8 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:39.720595 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81233893-f3e7-454a-8deb-425db7d5e5b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:39.722182 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81233893-f3e7-454a-8deb-425db7d5e5b0 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:39.725270 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:39.819318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c982a-62f9-4734-a526-32ade5e1fac8 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:39.846668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:39.849007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 168/342] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:39.850772 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81233893-f3e7-454a-8deb-425db7d5e5b0 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:39.859683 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4332c-ae88-4b54-a8c0-1f75fb7a67f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:39.860609 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4332c-ae88-4b54-a8c0-1f75fb7a67f1 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:39.861135 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4332c-ae88-4b54-a8c0-1f75fb7a67f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:39.861607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4332c-ae88-4b54-a8c0-1f75fb7a67f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:39.862539 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d4332c-ae88-4b54-a8c0-1f75fb7a67f1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:39.863049 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d4332c-ae88-4b54-a8c0-1f75fb7a67f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:39.864632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d4332c-ae88-4b54-a8c0-1f75fb7a67f1 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:39.875429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 175/343] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:22:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:39.881272 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4332c-ae88-4b54-a8c0-1f75fb7a67f1 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:39.995348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4332c-ae88-4b54-a8c0-1f75fb7a67f1 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:40.033476 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631ff05-5985-499f-b77e-673acc714250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:40.034479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631ff05-5985-499f-b77e-673acc714250 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:40.034953 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631ff05-5985-499f-b77e-673acc714250 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:40.035423 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631ff05-5985-499f-b77e-673acc714250 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:40.070231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:40.074724 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81233893-f3e7-454a-8deb-425db7d5e5b0 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:40.099923 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:40.114628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f631ff05-5985-499f-b77e-673acc714250 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:40.116400 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f631ff05-5985-499f-b77e-673acc714250 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:40.141625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a9836-e317-4bb7-a602-d53308a9dfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:40.142307 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a9836-e317-4bb7-a602-d53308a9dfd1 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:40.142600 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a9836-e317-4bb7-a602-d53308a9dfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:40.142849 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a9836-e317-4bb7-a602-d53308a9dfd1 None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:40.144057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220a9836-e317-4bb7-a602-d53308a9dfd1 None admin] RBAC: Authorizing `identity:get_project(project_id=5633253a5837491c995b7f11753765f8)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:40.145535 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220a9836-e317-4bb7-a602-d53308a9dfd1 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:40.147529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 169/344] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:39 2022] GET /identity/v3/projects/5633253a5837491c995b7f11753765f8 => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:40.153080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a9836-e317-4bb7-a602-d53308a9dfd1 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:40.239501 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 176/345] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 16:22:40.245502 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631ff05-5985-499f-b77e-673acc714250 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:40.361283 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:40.382149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631ff05-5985-499f-b77e-673acc714250 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:40.398641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a9836-e317-4bb7-a602-d53308a9dfd1 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:40.434370 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018de0e-2033-48de-9494-cd1d8d8d6f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:40.435233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018de0e-2033-48de-9494-cd1d8d8d6f2a None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:40.435611 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018de0e-2033-48de-9494-cd1d8d8d6f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:40.436290 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018de0e-2033-48de-9494-cd1d8d8d6f2a None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:40.437416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018de0e-2033-48de-9494-cd1d8d8d6f2a None admin] RBAC: Authorizing `identity:get_project(project_id=5633253a5837491c995b7f11753765f8)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:40.438928 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018de0e-2033-48de-9494-cd1d8d8d6f2a None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:40.441031 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 177/346] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:40 2022] GET /identity/v3/projects/5633253a5837491c995b7f11753765f8 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:40.446758 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018de0e-2033-48de-9494-cd1d8d8d6f2a None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:40.457672 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fada33-b5f8-4257-aa09-f8ca37d16fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:40.458551 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fada33-b5f8-4257-aa09-f8ca37d16fef None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:40.458994 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fada33-b5f8-4257-aa09-f8ca37d16fef None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:40.459409 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fada33-b5f8-4257-aa09-f8ca37d16fef None admin] PATH_INFO: `/v3/groups/a21ac91a329948a2afa5d30d7e107040` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:40.477586 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fada33-b5f8-4257-aa09-f8ca37d16fef None admin] RBAC: Authorizing `identity:get_group(group_id=a21ac91a329948a2afa5d30d7e107040)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:40.479280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fada33-b5f8-4257-aa09-f8ca37d16fef None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:40.481991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 170/347] 10.222.0.74 () {56 vars in 1180 bytes} [Wed Jul 27 16:22:40 2022] GET /identity/v3/groups/a21ac91a329948a2afa5d30d7e107040 => generated 226 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:40.493325 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fada33-b5f8-4257-aa09-f8ca37d16fef None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:40.530666 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:40.606972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fada33-b5f8-4257-aa09-f8ca37d16fef None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:40.645410 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffdc9b-ea17-4636-b429-794b73742f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:40.646324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffdc9b-ea17-4636-b429-794b73742f1e None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:40.646769 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffdc9b-ea17-4636-b429-794b73742f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:40.647232 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffdc9b-ea17-4636-b429-794b73742f1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:40.648184 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feffdc9b-ea17-4636-b429-794b73742f1e None admin] RBAC: Adding query filter params (group.id=a21ac91a329948a2afa5d30d7e107040, role.id=8de48592893c43789e1d40158943fd52, scope.project.id=5633253a5837491c995b7f11753765f8) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:40.648676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feffdc9b-ea17-4636-b429-794b73742f1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:40.650196 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018de0e-2033-48de-9494-cd1d8d8d6f2a None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:40.650521 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feffdc9b-ea17-4636-b429-794b73742f1e None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:40.656925 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 171/348] 10.222.0.74 () {56 vars in 1376 bytes} [Wed Jul 27 16:22:40 2022] GET /identity/v3/role_assignments?group.id=a21ac91a329948a2afa5d30d7e107040&scope.project.id=5633253a5837491c995b7f11753765f8&role.id=8de48592893c43789e1d40158943fd52 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:40.658492 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5baa1d-6293-416e-9d35-64457b3ba422 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:40.659120 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5baa1d-6293-416e-9d35-64457b3ba422 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:40.659595 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5baa1d-6293-416e-9d35-64457b3ba422 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:40.661934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 172/349] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:40.667671 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7ca2e-db88-45ad-8762-f104b95d6ae8 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:40.668222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7ca2e-db88-45ad-8762-f104b95d6ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:40.668795 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7ca2e-db88-45ad-8762-f104b95d6ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:40.711305 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-0bd7ca2e-db88-45ad-8762-f104b95d6ae8 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:40.713105 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0bd7ca2e-db88-45ad-8762-f104b95d6ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:40.717672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312eb41-18f4-4c7f-9b38-6b36f8405d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:40.718354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312eb41-18f4-4c7f-9b38-6b36f8405d33 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:40.718633 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312eb41-18f4-4c7f-9b38-6b36f8405d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:40.718903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312eb41-18f4-4c7f-9b38-6b36f8405d33 None admin] PATH_INFO: `/v3/groups/a21ac91a329948a2afa5d30d7e107040` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:40.720134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312eb41-18f4-4c7f-9b38-6b36f8405d33 None admin] RBAC: Authorizing `identity:get_group(group_id=a21ac91a329948a2afa5d30d7e107040)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:40.724585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312eb41-18f4-4c7f-9b38-6b36f8405d33 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:40.724585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 178/350] 10.222.0.74 () {56 vars in 1180 bytes} [Wed Jul 27 16:22:40 2022] GET /identity/v3/groups/a21ac91a329948a2afa5d30d7e107040 => generated 226 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:40.729348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312eb41-18f4-4c7f-9b38-6b36f8405d33 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:40.772094 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:40.818554 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 173/351] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:40.820265 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3135578-f6cb-4ae9-8447-d9b6d662d2af None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:40.820766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3135578-f6cb-4ae9-8447-d9b6d662d2af None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:40.821286 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3135578-f6cb-4ae9-8447-d9b6d662d2af None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:40.823188 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 174/352] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:40.829272 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb937489-3ba2-4ddb-b56b-40154c3def1e None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:40.829860 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb937489-3ba2-4ddb-b56b-40154c3def1e None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:40.830357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb937489-3ba2-4ddb-b56b-40154c3def1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:40.861894 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312eb41-18f4-4c7f-9b38-6b36f8405d33 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:40.862291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-fb937489-3ba2-4ddb-b56b-40154c3def1e None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:40.863971 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-fb937489-3ba2-4ddb-b56b-40154c3def1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:40.895158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:40.901841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e653fc-d5d0-4744-add3-5c2556ba4655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:40.902495 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e653fc-d5d0-4744-add3-5c2556ba4655 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:40.903382 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e653fc-d5d0-4744-add3-5c2556ba4655 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:40.903663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e653fc-d5d0-4744-add3-5c2556ba4655 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:40.904681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e653fc-d5d0-4744-add3-5c2556ba4655 None admin] RBAC: Adding query filter params (group.id=a21ac91a329948a2afa5d30d7e107040, role.id=70f3e09bc103410789de8e15c81d9cd3, scope.project.id=5633253a5837491c995b7f11753765f8) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:40.908914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e653fc-d5d0-4744-add3-5c2556ba4655 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:40.908914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e653fc-d5d0-4744-add3-5c2556ba4655 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:40.913003 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 179/353] 10.222.0.74 () {56 vars in 1376 bytes} [Wed Jul 27 16:22:40 2022] GET /identity/v3/role_assignments?group.id=a21ac91a329948a2afa5d30d7e107040&scope.project.id=5633253a5837491c995b7f11753765f8&role.id=70f3e09bc103410789de8e15c81d9cd3 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:40.924919 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e653fc-d5d0-4744-add3-5c2556ba4655 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:40.989240 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 175/354] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:41.004650 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ab7b5-e153-4146-9ce7-187d79fa970b None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:41.005296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ab7b5-e153-4146-9ce7-187d79fa970b None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:41.005808 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ab7b5-e153-4146-9ce7-187d79fa970b None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:41.007679 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 176/355] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:22:41.014550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67be05-6371-415f-a028-cf467182551b None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:41.015171 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67be05-6371-415f-a028-cf467182551b None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:41.015643 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67be05-6371-415f-a028-cf467182551b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:41.036836 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-eb67be05-6371-415f-a028-cf467182551b None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:41.038462 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-eb67be05-6371-415f-a028-cf467182551b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:41.041669 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:41.049504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e653fc-d5d0-4744-add3-5c2556ba4655 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:41.091655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77867103-2317-4564-80af-f1ced480192a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:41.092233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77867103-2317-4564-80af-f1ced480192a None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:41.092521 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77867103-2317-4564-80af-f1ced480192a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:41.092796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77867103-2317-4564-80af-f1ced480192a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:41.104305 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77867103-2317-4564-80af-f1ced480192a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:41.110947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-77867103-2317-4564-80af-f1ced480192a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:41.110947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:41.110947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:41.110947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:41.110947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:41.110947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:41.110947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:41.110947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:41.110947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:41.110947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:41.110947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:41.110947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:41.110947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:41.110947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:41.110947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:41.110947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:41.111899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:41.111899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:41.111899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:41.111899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:41.111899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:41.111899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:41.111899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:41.111899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:41.111899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:41.111899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:41.111899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:41.111899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:41.111899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:41.111899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:41.111899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:41.111899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:41.112699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:41.112699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:41.112699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:41.112699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:41.112699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:41.112699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:41.112699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:41.112699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:41.112699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:41.112699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:41.112699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:41.112699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:41.112699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 180/356] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:41.114126 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77867103-2317-4564-80af-f1ced480192a None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:41.123673 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 177/357] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:41.129296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67be05-6371-415f-a028-cf467182551b None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:41.241338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:41.243030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77867103-2317-4564-80af-f1ced480192a None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:41.259815 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67be05-6371-415f-a028-cf467182551b None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:41.282713 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042aa59-241a-4bf9-8d0f-732f8be1b4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:41.283614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042aa59-241a-4bf9-8d0f-732f8be1b4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:41.284044 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042aa59-241a-4bf9-8d0f-732f8be1b4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:41.284468 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042aa59-241a-4bf9-8d0f-732f8be1b4b2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:41.296920 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4042aa59-241a-4bf9-8d0f-732f8be1b4b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:41.297266 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286ae49-3c2c-4e0f-b338-b193e2164d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:41.298023 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286ae49-3c2c-4e0f-b338-b193e2164d2e None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:41.298405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286ae49-3c2c-4e0f-b338-b193e2164d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:41.298780 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286ae49-3c2c-4e0f-b338-b193e2164d2e None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:41.299614 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286ae49-3c2c-4e0f-b338-b193e2164d2e None admin] RBAC: Adding query filter params (name=member) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:41.300674 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286ae49-3c2c-4e0f-b338-b193e2164d2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:41.302261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286ae49-3c2c-4e0f-b338-b193e2164d2e None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:41.304661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-4042aa59-241a-4bf9-8d0f-732f8be1b4b2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 16:22:41.304661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:41.304661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:41.304661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:41.304661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:41.304661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:41.304661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:41.304661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:41.304661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:41.304661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:41.304661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:41.304661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:41.304661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:41.304661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:41.304661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:41.304661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:41.305582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:41.305582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:41.305582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:41.305582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:41.305582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:41.305582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:41.305582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:41.305582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:41.305582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:41.305582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:41.305582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:41.305582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:41.305582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:41.305582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:41.305582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:41.305582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:41.306380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:41.306380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:41.306380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:41.306380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:41.306380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:41.306380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:41.306380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:41.306380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:41.306380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:41.306380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:41.306380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 16:22:41.306380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:41.308813 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 181/358] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:41.310426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042aa59-241a-4bf9-8d0f-732f8be1b4b2 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:41.313768 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 178/359] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:41.317971 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286ae49-3c2c-4e0f-b338-b193e2164d2e None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:41.341290 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:41.425441 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042aa59-241a-4bf9-8d0f-732f8be1b4b2 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:41.430181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286ae49-3c2c-4e0f-b338-b193e2164d2e None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:41.464121 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fa6cb-17be-4504-875a-11d8a4979028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:41.464792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fa6cb-17be-4504-875a-11d8a4979028 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:41.465104 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fa6cb-17be-4504-875a-11d8a4979028 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:41.466299 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fa6cb-17be-4504-875a-11d8a4979028 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:41.470175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e6dd5-7444-447a-a94d-e76365fb8471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:41.470902 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e6dd5-7444-447a-a94d-e76365fb8471 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:41.471255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e6dd5-7444-447a-a94d-e76365fb8471 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:41.471601 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e6dd5-7444-447a-a94d-e76365fb8471 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:41.472393 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e6dd5-7444-447a-a94d-e76365fb8471 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:41.472985 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e6dd5-7444-447a-a94d-e76365fb8471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:41.474528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e6dd5-7444-447a-a94d-e76365fb8471 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:41.479086 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fa6cb-17be-4504-875a-11d8a4979028 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:41.485835 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-de7fa6cb-17be-4504-875a-11d8a4979028 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:41.485835 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:41.485835 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:41.485835 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:41.485835 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:41.485835 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:41.485835 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:41.485835 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:41.485835 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:41.485835 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:41.485835 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:41.485835 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:41.485835 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:41.485835 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:41.485835 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:41.485835 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:41.486901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:41.486901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:41.486901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:41.486901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:41.486901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:41.486901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:41.486901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:41.486901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:41.486901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:41.486901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:41.486901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:41.486901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:41.486901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:41.486901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:41.486901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:41.486901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:41.487942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:41.487942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:41.487942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:41.487942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:41.487942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:41.487942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:41.487942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:41.487942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:41.487942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:41.487942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:41.487942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:41.487942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:41.487942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 182/360] 10.222.0.74 () {56 vars in 1114 bytes} [Wed Jul 27 16:22:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:41.488732 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fa6cb-17be-4504-875a-11d8a4979028 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:41.493531 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 179/361] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:41.495736 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e6dd5-7444-447a-a94d-e76365fb8471 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:41.509827 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:41.581252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:41.642565 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fa6cb-17be-4504-875a-11d8a4979028 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:41.653782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e6dd5-7444-447a-a94d-e76365fb8471 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:41.707858 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075e336-3e2c-44e5-986c-ba227d97a5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:41.708488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075e336-3e2c-44e5-986c-ba227d97a5cb None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:41.708769 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075e336-3e2c-44e5-986c-ba227d97a5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:41.710669 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075e336-3e2c-44e5-986c-ba227d97a5cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:41.717316 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d705686-8779-4da5-973b-aa31121d4119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:41.717949 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d705686-8779-4da5-973b-aa31121d4119 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:41.718231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d705686-8779-4da5-973b-aa31121d4119 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:41.718506 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d705686-8779-4da5-973b-aa31121d4119 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:41.719231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d705686-8779-4da5-973b-aa31121d4119 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:41.719535 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d705686-8779-4da5-973b-aa31121d4119 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:41.720938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d705686-8779-4da5-973b-aa31121d4119 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:41.739128 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 180/362] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:22:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:41.739759 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d705686-8779-4da5-973b-aa31121d4119 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:41.853319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5075e336-3e2c-44e5-986c-ba227d97a5cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:41.854790 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5075e336-3e2c-44e5-986c-ba227d97a5cb None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:41.892729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:41.944467 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d705686-8779-4da5-973b-aa31121d4119 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:41.985429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:41.993229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7195234-aba0-4cd1-aa58-b2adfa595e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:41.993994 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7195234-aba0-4cd1-aa58-b2adfa595e4c None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:41.994355 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7195234-aba0-4cd1-aa58-b2adfa595e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:41.994730 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7195234-aba0-4cd1-aa58-b2adfa595e4c None admin] PATH_INFO: `/v3/users/658f5cb042a1480b9a1fc58d44ce78bc` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:41.996044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7195234-aba0-4cd1-aa58-b2adfa595e4c None admin] RBAC: Authorizing `identity:get_user(user_id=658f5cb042a1480b9a1fc58d44ce78bc)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:41.998560 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7195234-aba0-4cd1-aa58-b2adfa595e4c None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:42.001271 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 181/363] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:22:41 2022] GET /identity/v3/users/658f5cb042a1480b9a1fc58d44ce78bc => generated 293 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:42.002415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7195234-aba0-4cd1-aa58-b2adfa595e4c None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:42.030185 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 183/364] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:42.031470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1ae1c-3688-4b38-88ed-540c6fa2dc93 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:42.031762 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1ae1c-3688-4b38-88ed-540c6fa2dc93 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:42.032041 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1ae1c-3688-4b38-88ed-540c6fa2dc93 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:42.034104 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 184/365] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:42.034874 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1ae1c-3688-4b38-88ed-540c6fa2dc93 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:42.138079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7195234-aba0-4cd1-aa58-b2adfa595e4c None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:42.175135 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1ae1c-3688-4b38-88ed-540c6fa2dc93 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:42.185447 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4f125-9de5-4fd4-ad0b-e14e42d7342a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:42.186286 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4f125-9de5-4fd4-ad0b-e14e42d7342a None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:42.186719 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4f125-9de5-4fd4-ad0b-e14e42d7342a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:42.187563 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4f125-9de5-4fd4-ad0b-e14e42d7342a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:42.201459 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:42.221937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1124b-2fa5-4063-b76d-f2b554c2354f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:42.222651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1124b-2fa5-4063-b76d-f2b554c2354f None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:42.223007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1124b-2fa5-4063-b76d-f2b554c2354f None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:42.223343 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1124b-2fa5-4063-b76d-f2b554c2354f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:42.224294 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1124b-2fa5-4063-b76d-f2b554c2354f None admin] RBAC: Adding query filter params (role.id=1204be9fc6d7483fb28ba04dfa5b3f6e, user.id=658f5cb042a1480b9a1fc58d44ce78bc, scope.system=all) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:42.224668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1124b-2fa5-4063-b76d-f2b554c2354f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:42.226311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1124b-2fa5-4063-b76d-f2b554c2354f None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:42.240800 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 185/366] 10.222.0.74 () {56 vars in 1308 bytes} [Wed Jul 27 16:22:42 2022] GET /identity/v3/role_assignments?user.id=658f5cb042a1480b9a1fc58d44ce78bc&scope.system=all&role.id=1204be9fc6d7483fb28ba04dfa5b3f6e => generated 510 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:42.241921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1124b-2fa5-4063-b76d-f2b554c2354f None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:42.276322 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:42.279091 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4f125-9de5-4fd4-ad0b-e14e42d7342a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:42.280789 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4f125-9de5-4fd4-ad0b-e14e42d7342a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:42.368665 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1124b-2fa5-4063-b76d-f2b554c2354f None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:42.408497 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eeb0fd-86fb-488e-b4b9-a3c6d9cf08ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:42.410892 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eeb0fd-86fb-488e-b4b9-a3c6d9cf08ae None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:42.412454 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eeb0fd-86fb-488e-b4b9-a3c6d9cf08ae None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:42.413078 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eeb0fd-86fb-488e-b4b9-a3c6d9cf08ae None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:42.416864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eeb0fd-86fb-488e-b4b9-a3c6d9cf08ae None admin] RBAC: Authorizing `identity:get_project(project_id=6a66a60a14e247e5bf2142da6f43cc44)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:42.424123 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eeb0fd-86fb-488e-b4b9-a3c6d9cf08ae None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:42.426387 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 186/367] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:42 2022] GET /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:42.427726 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d4457-cba9-4620-8f5c-aae16757c6b7 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:42.430256 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d4457-cba9-4620-8f5c-aae16757c6b7 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:42.430687 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d4457-cba9-4620-8f5c-aae16757c6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:42.465633 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-2a7d4457-cba9-4620-8f5c-aae16757c6b7 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:42.467460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-2a7d4457-cba9-4620-8f5c-aae16757c6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:42.484029 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 182/368] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:42.485321 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4f125-9de5-4fd4-ad0b-e14e42d7342a None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:42.532189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:42.595794 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 187/369] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:42.597025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d4457-cba9-4620-8f5c-aae16757c6b7 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:42.610828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4f125-9de5-4fd4-ad0b-e14e42d7342a None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:42.629899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:42.658523 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a6a73-65cb-440e-aca5-021374fcf088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:42.659199 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a6a73-65cb-440e-aca5-021374fcf088 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:42.659484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a6a73-65cb-440e-aca5-021374fcf088 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:42.659762 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a6a73-65cb-440e-aca5-021374fcf088 None admin] PATH_INFO: `/v3/groups/a21ac91a329948a2afa5d30d7e107040` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:42.661054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a6a73-65cb-440e-aca5-021374fcf088 None admin] RBAC: Authorizing `identity:get_group(group_id=a21ac91a329948a2afa5d30d7e107040)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:42.662650 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a6a73-65cb-440e-aca5-021374fcf088 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:42.664321 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 183/370] 10.222.0.74 () {56 vars in 1180 bytes} [Wed Jul 27 16:22:42 2022] GET /identity/v3/groups/a21ac91a329948a2afa5d30d7e107040 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:42.665269 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a6a73-65cb-440e-aca5-021374fcf088 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:42.742653 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d4457-cba9-4620-8f5c-aae16757c6b7 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:42.781287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c625d-6ff0-4c91-8fae-8846bb2d0733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:42.782442 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c625d-6ff0-4c91-8fae-8846bb2d0733 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:42.783023 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c625d-6ff0-4c91-8fae-8846bb2d0733 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:42.783513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c625d-6ff0-4c91-8fae-8846bb2d0733 None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:42.785551 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c625d-6ff0-4c91-8fae-8846bb2d0733 None admin] RBAC: Authorizing `identity:get_project(project_id=6a66a60a14e247e5bf2142da6f43cc44)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:42.787328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c625d-6ff0-4c91-8fae-8846bb2d0733 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:42.789691 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 188/371] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:42 2022] GET /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:42.790784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c625d-6ff0-4c91-8fae-8846bb2d0733 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:42.804094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a6a73-65cb-440e-aca5-021374fcf088 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:42.814117 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:42.850287 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa145e5d-8862-427a-a51a-d767f60860e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:42.850946 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa145e5d-8862-427a-a51a-d767f60860e5 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:42.851403 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa145e5d-8862-427a-a51a-d767f60860e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:42.851779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa145e5d-8862-427a-a51a-d767f60860e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:42.865770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa145e5d-8862-427a-a51a-d767f60860e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:42.872630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-aa145e5d-8862-427a-a51a-d767f60860e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:22:42.872630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:42.872630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:42.872630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:42.872630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:42.872630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:42.872630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:42.872630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:42.872630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:42.872630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:42.872630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:42.872630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:42.872630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:42.872630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:42.872630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:42.872630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:42.873559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:42.873559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:42.873559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:42.873559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:42.873559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:42.873559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:42.873559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:42.873559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:42.873559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:42.873559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:42.873559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:42.873559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:42.873559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:42.873559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:42.873559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:42.873559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:42.874351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:42.874351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:42.874351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:42.874351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:42.874351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:42.874351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:42.874351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:42.874351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:42.874351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:42.874351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:42.874351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:22:42.874351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:42.874351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 184/372] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:22:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:42.875073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa145e5d-8862-427a-a51a-d767f60860e5 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:42.909995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:42.922113 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c625d-6ff0-4c91-8fae-8846bb2d0733 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:42.963857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af290b-9b8a-4c4e-b958-8c33fff7dc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:42.964983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af290b-9b8a-4c4e-b958-8c33fff7dc55 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:42.965998 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af290b-9b8a-4c4e-b958-8c33fff7dc55 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:42.966548 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af290b-9b8a-4c4e-b958-8c33fff7dc55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:42.967536 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af290b-9b8a-4c4e-b958-8c33fff7dc55 None admin] RBAC: Adding query filter params (group.id=a21ac91a329948a2afa5d30d7e107040, role.id=8de48592893c43789e1d40158943fd52, scope.project.id=6a66a60a14e247e5bf2142da6f43cc44) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:42.968099 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af290b-9b8a-4c4e-b958-8c33fff7dc55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:42.970017 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af290b-9b8a-4c4e-b958-8c33fff7dc55 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:42.977509 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 189/373] 10.222.0.74 () {56 vars in 1376 bytes} [Wed Jul 27 16:22:42 2022] GET /identity/v3/role_assignments?group.id=a21ac91a329948a2afa5d30d7e107040&scope.project.id=6a66a60a14e247e5bf2142da6f43cc44&role.id=8de48592893c43789e1d40158943fd52 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:42.978789 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af290b-9b8a-4c4e-b958-8c33fff7dc55 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:43.011075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa145e5d-8862-427a-a51a-d767f60860e5 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:43.056989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14ef52-e313-405d-aac8-bc56d6b17139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:43.057524 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14ef52-e313-405d-aac8-bc56d6b17139 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:43.057809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14ef52-e313-405d-aac8-bc56d6b17139 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:43.058086 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14ef52-e313-405d-aac8-bc56d6b17139 None admin] PATH_INFO: `/v3/groups/a21ac91a329948a2afa5d30d7e107040` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:43.061884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b14ef52-e313-405d-aac8-bc56d6b17139 None admin] RBAC: Authorizing `identity:get_group(group_id=a21ac91a329948a2afa5d30d7e107040)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:43.061884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b14ef52-e313-405d-aac8-bc56d6b17139 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:43.064515 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 185/374] 10.222.0.74 () {56 vars in 1180 bytes} [Wed Jul 27 16:22:42 2022] GET /identity/v3/groups/a21ac91a329948a2afa5d30d7e107040 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:43.075581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14ef52-e313-405d-aac8-bc56d6b17139 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:43.151546 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af290b-9b8a-4c4e-b958-8c33fff7dc55 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:43.153224 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:43.232293 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a9a47-18e1-493b-96bc-a27a3092bd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:43.233017 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a9a47-18e1-493b-96bc-a27a3092bd87 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:43.233331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a9a47-18e1-493b-96bc-a27a3092bd87 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:43.233607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a9a47-18e1-493b-96bc-a27a3092bd87 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:43.234335 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a9a47-18e1-493b-96bc-a27a3092bd87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:43.234641 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a9a47-18e1-493b-96bc-a27a3092bd87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:43.241064 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a9a47-18e1-493b-96bc-a27a3092bd87 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:43.252711 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 190/375] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:43.261922 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a9a47-18e1-493b-96bc-a27a3092bd87 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:43.268344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14ef52-e313-405d-aac8-bc56d6b17139 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:43.302904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:43.316293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6586659-018c-48f1-bf95-7b7bf5d543ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:43.318106 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6586659-018c-48f1-bf95-7b7bf5d543ef None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:43.318940 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6586659-018c-48f1-bf95-7b7bf5d543ef None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:43.319666 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6586659-018c-48f1-bf95-7b7bf5d543ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:43.321584 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6586659-018c-48f1-bf95-7b7bf5d543ef None admin] RBAC: Adding query filter params (group.id=a21ac91a329948a2afa5d30d7e107040, role.id=70f3e09bc103410789de8e15c81d9cd3, scope.project.id=6a66a60a14e247e5bf2142da6f43cc44) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:43.322131 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6586659-018c-48f1-bf95-7b7bf5d543ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:43.324301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6586659-018c-48f1-bf95-7b7bf5d543ef None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:43.332439 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 186/376] 10.222.0.74 () {56 vars in 1376 bytes} [Wed Jul 27 16:22:43 2022] GET /identity/v3/role_assignments?group.id=a21ac91a329948a2afa5d30d7e107040&scope.project.id=6a66a60a14e247e5bf2142da6f43cc44&role.id=70f3e09bc103410789de8e15c81d9cd3 => generated 260 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:43.452332 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a9a47-18e1-493b-96bc-a27a3092bd87 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:43.462156 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874034c-13c8-48e4-9de2-9b34747e3fff None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:43.462487 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874034c-13c8-48e4-9de2-9b34747e3fff None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:43.464296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874034c-13c8-48e4-9de2-9b34747e3fff None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:43.464409 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 187/377] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:43.478418 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112721f2-e259-4b1c-b314-c8f4487ec8ec None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:43.478728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112721f2-e259-4b1c-b314-c8f4487ec8ec None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:43.478982 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112721f2-e259-4b1c-b314-c8f4487ec8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:43.498217 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa90a7-5d21-4a7c-86e8-5c202ac72ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:43.498854 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa90a7-5d21-4a7c-86e8-5c202ac72ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:43.499133 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa90a7-5d21-4a7c-86e8-5c202ac72ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:43.499405 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa90a7-5d21-4a7c-86e8-5c202ac72ef8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:43.511008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-112721f2-e259-4b1c-b314-c8f4487ec8ec None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:43.518611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-112721f2-e259-4b1c-b314-c8f4487ec8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:43.584518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa90a7-5d21-4a7c-86e8-5c202ac72ef8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:43.586327 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa90a7-5d21-4a7c-86e8-5c202ac72ef8 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:43.605940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:43.625399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 188/378] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:43.645486 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112721f2-e259-4b1c-b314-c8f4487ec8ec None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:43.677023 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:43.695255 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 191/379] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:43.701291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa90a7-5d21-4a7c-86e8-5c202ac72ef8 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:43.757491 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112721f2-e259-4b1c-b314-c8f4487ec8ec None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:43.798145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee0c31-b1a5-4f32-a08f-3e8f0ded96ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:43.798801 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee0c31-b1a5-4f32-a08f-3e8f0ded96ca None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:43.799081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee0c31-b1a5-4f32-a08f-3e8f0ded96ca None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:43.799352 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee0c31-b1a5-4f32-a08f-3e8f0ded96ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:43.807589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa90a7-5d21-4a7c-86e8-5c202ac72ef8 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:43.811323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee0c31-b1a5-4f32-a08f-3e8f0ded96ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:43.816402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-92ee0c31-b1a5-4f32-a08f-3e8f0ded96ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:43.816402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:43.816402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:43.816402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:43.816402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:43.816402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:43.816402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:43.816402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:43.816402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:43.816402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:43.816402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:43.816402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:43.816402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:43.816402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:43.816402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:43.816402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:43.817299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:43.817299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:43.817299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:43.817299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:43.817299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:43.817299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:43.817299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:43.817299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:43.817299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:43.817299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:43.817299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:43.817299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:43.817299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:43.817299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:43.817299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:43.817299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:43.818081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:43.818081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:43.818081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:43.818081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:43.818081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:43.818081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:43.818081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:43.818081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:43.818081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:43.818081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:43.818081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:43.818081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:43.821308 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 189/380] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:22:43.824112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee0c31-b1a5-4f32-a08f-3e8f0ded96ca None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:43.840992 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:43.868246 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96becdc8-a776-47e7-8c87-c0962d8e386f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:43.869309 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96becdc8-a776-47e7-8c87-c0962d8e386f None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:43.869809 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96becdc8-a776-47e7-8c87-c0962d8e386f None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:43.870326 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96becdc8-a776-47e7-8c87-c0962d8e386f None admin] PATH_INFO: `/v3/projects/0d94f0c6f55c441591df5c2cef40a859` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:43.877659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96becdc8-a776-47e7-8c87-c0962d8e386f None admin] RBAC: Authorizing `identity:get_project(project_id=0d94f0c6f55c441591df5c2cef40a859)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:43.879369 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96becdc8-a776-47e7-8c87-c0962d8e386f None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:43.881662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 192/381] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:43 2022] GET /identity/v3/projects/0d94f0c6f55c441591df5c2cef40a859 => generated 344 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:43.886893 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96becdc8-a776-47e7-8c87-c0962d8e386f None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:43.935814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee0c31-b1a5-4f32-a08f-3e8f0ded96ca None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:43.951499 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:43.974275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54af802-8562-427e-b34d-886577a66372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:43.975006 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54af802-8562-427e-b34d-886577a66372 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:43.975283 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54af802-8562-427e-b34d-886577a66372 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:43.975484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54af802-8562-427e-b34d-886577a66372 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:43.976205 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54af802-8562-427e-b34d-886577a66372 None admin] RBAC: Adding query filter params (name=member) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:43.976618 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54af802-8562-427e-b34d-886577a66372 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:43.978027 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54af802-8562-427e-b34d-886577a66372 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:43.992638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 190/382] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:43.993919 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54af802-8562-427e-b34d-886577a66372 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:44.077026 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96becdc8-a776-47e7-8c87-c0962d8e386f None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:44.102146 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54af802-8562-427e-b34d-886577a66372 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:44.129451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d1708-83c6-49ca-879e-2477138daa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:44.130501 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d1708-83c6-49ca-879e-2477138daa13 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:44.130854 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d1708-83c6-49ca-879e-2477138daa13 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:44.131206 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d1708-83c6-49ca-879e-2477138daa13 None admin] PATH_INFO: `/v3/groups/dad3e0411f804865bb09225e0ee66f55` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:44.140171 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d1708-83c6-49ca-879e-2477138daa13 None admin] RBAC: Authorizing `identity:get_group(group_id=dad3e0411f804865bb09225e0ee66f55)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:44.142170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d1708-83c6-49ca-879e-2477138daa13 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:44.142280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f4612-8c77-46a0-90d3-94f45e0bf4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:44.142835 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f4612-8c77-46a0-90d3-94f45e0bf4c9 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:44.143137 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f4612-8c77-46a0-90d3-94f45e0bf4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:44.143416 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f4612-8c77-46a0-90d3-94f45e0bf4c9 None admin] PATH_INFO: `/v3/groups/a21ac91a329948a2afa5d30d7e107040` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:44.144565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f4612-8c77-46a0-90d3-94f45e0bf4c9 None admin] RBAC: Authorizing `identity:get_group(group_id=a21ac91a329948a2afa5d30d7e107040)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:44.145430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 193/383] 10.222.0.74 () {56 vars in 1180 bytes} [Wed Jul 27 16:22:43 2022] GET /identity/v3/groups/dad3e0411f804865bb09225e0ee66f55 => generated 229 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:44.146163 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f4612-8c77-46a0-90d3-94f45e0bf4c9 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:44.147018 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c460237-6777-4931-85e7-24316421ce4c None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:44.147505 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c460237-6777-4931-85e7-24316421ce4c None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:44.148003 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c460237-6777-4931-85e7-24316421ce4c None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:44.153392 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 191/384] 10.222.0.74 () {56 vars in 1180 bytes} [Wed Jul 27 16:22:43 2022] GET /identity/v3/groups/a21ac91a329948a2afa5d30d7e107040 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:44.154153 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f4612-8c77-46a0-90d3-94f45e0bf4c9 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:44.154380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 194/385] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:44.156775 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c460237-6777-4931-85e7-24316421ce4c None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:44.183467 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:44.223149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:44.277953 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f4612-8c77-46a0-90d3-94f45e0bf4c9 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:44.293943 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c460237-6777-4931-85e7-24316421ce4c None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:44.333006 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de0148-4360-425b-9cfc-385f8f0cf252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:44.335011 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de0148-4360-425b-9cfc-385f8f0cf252 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:44.336097 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de0148-4360-425b-9cfc-385f8f0cf252 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:44.336787 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de0148-4360-425b-9cfc-385f8f0cf252 None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:44.338900 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de0148-4360-425b-9cfc-385f8f0cf252 None admin] RBAC: Authorizing `identity:get_project(project_id=5633253a5837491c995b7f11753765f8)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:44.341662 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de0148-4360-425b-9cfc-385f8f0cf252 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:44.342742 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed43d7-87f9-4d51-8de1-cea4dd7d934d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:44.343368 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed43d7-87f9-4d51-8de1-cea4dd7d934d None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:44.343652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed43d7-87f9-4d51-8de1-cea4dd7d934d None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:44.343929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed43d7-87f9-4d51-8de1-cea4dd7d934d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:44.344728 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed43d7-87f9-4d51-8de1-cea4dd7d934d None admin] RBAC: Adding query filter params (group.id=dad3e0411f804865bb09225e0ee66f55, role.id=ca37e43082fa40c6a88b12680a0ca94b, scope.project.id=0d94f0c6f55c441591df5c2cef40a859) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:44.345089 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed43d7-87f9-4d51-8de1-cea4dd7d934d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:44.346688 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed43d7-87f9-4d51-8de1-cea4dd7d934d None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:44.347257 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 192/386] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:44 2022] GET /identity/v3/projects/5633253a5837491c995b7f11753765f8 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:44.350926 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11518830-6cab-4475-a4eb-1d8c12281cbd None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:44.352106 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11518830-6cab-4475-a4eb-1d8c12281cbd None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:44.353052 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11518830-6cab-4475-a4eb-1d8c12281cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:44.353888 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 195/387] 10.222.0.74 () {56 vars in 1376 bytes} [Wed Jul 27 16:22:44 2022] GET /identity/v3/role_assignments?group.id=dad3e0411f804865bb09225e0ee66f55&scope.project.id=0d94f0c6f55c441591df5c2cef40a859&role.id=ca37e43082fa40c6a88b12680a0ca94b => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:44.354810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed43d7-87f9-4d51-8de1-cea4dd7d934d None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:44.390806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-11518830-6cab-4475-a4eb-1d8c12281cbd None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:44.392034 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-11518830-6cab-4475-a4eb-1d8c12281cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:44.506288 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 193/388] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:44.537046 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11518830-6cab-4475-a4eb-1d8c12281cbd None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:44.558640 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:44.598757 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed43d7-87f9-4d51-8de1-cea4dd7d934d None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:44.619292 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:44.658483 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45368e6-13b1-4b2e-96c7-13f53f575dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:44.659550 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45368e6-13b1-4b2e-96c7-13f53f575dd5 None admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:44.660039 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45368e6-13b1-4b2e-96c7-13f53f575dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:44.660543 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45368e6-13b1-4b2e-96c7-13f53f575dd5 None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8/groups/a21ac91a329948a2afa5d30d7e107040/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:44.662748 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45368e6-13b1-4b2e-96c7-13f53f575dd5 None admin] RBAC: Authorizing `identity:create_grant(project_id=5633253a5837491c995b7f11753765f8, group_id=a21ac91a329948a2afa5d30d7e107040, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:44.664413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45368e6-13b1-4b2e-96c7-13f53f575dd5 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:44.691681 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11518830-6cab-4475-a4eb-1d8c12281cbd None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:44.695962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 196/389] 10.222.0.74 () {58 vars in 1442 bytes} [Wed Jul 27 16:22:44 2022] PUT /identity/v3/projects/5633253a5837491c995b7f11753765f8/groups/a21ac91a329948a2afa5d30d7e107040/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:22:44.743649 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966e2f0-3070-4113-9719-aaf59ad77012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:44.744379 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966e2f0-3070-4113-9719-aaf59ad77012 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:44.744660 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966e2f0-3070-4113-9719-aaf59ad77012 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:44.744956 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966e2f0-3070-4113-9719-aaf59ad77012 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:44.757938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966e2f0-3070-4113-9719-aaf59ad77012 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:44.763094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-a966e2f0-3070-4113-9719-aaf59ad77012 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:44.763094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:44.763094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:44.763094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:44.763094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:44.763094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:44.763094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:44.763094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:44.763094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:44.763094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:44.763094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:44.763094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:44.763094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:44.763094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:44.763094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:44.763094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:44.764076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:44.764076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:44.764076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:44.764076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:44.764076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:44.764076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:44.764076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:44.764076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:44.764076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:44.764076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:44.764076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:44.764076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:44.764076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:44.764076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:44.764076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:44.764076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:44.764972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:44.764972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:44.764972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:44.764972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:44.764972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:44.764972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:44.764972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:44.764972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:44.764972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:44.764972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:44.764972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:44.764972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:44.764972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 194/390] 10.222.0.74 () {56 vars in 1114 bytes} [Wed Jul 27 16:22:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:44.770699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45368e6-13b1-4b2e-96c7-13f53f575dd5 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:44.930075 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45368e6-13b1-4b2e-96c7-13f53f575dd5 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:44.966304 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e5d41-1021-4db0-b3ea-976b69786cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:44.966941 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e5d41-1021-4db0-b3ea-976b69786cda None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:44.967225 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e5d41-1021-4db0-b3ea-976b69786cda None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:44.967558 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e5d41-1021-4db0-b3ea-976b69786cda None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:44.968290 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e5d41-1021-4db0-b3ea-976b69786cda None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:44.968599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e5d41-1021-4db0-b3ea-976b69786cda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:44.970017 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e5d41-1021-4db0-b3ea-976b69786cda None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:44.979036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 197/391] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:22:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:44.985205 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966e2f0-3070-4113-9719-aaf59ad77012 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:45.088623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:45.126671 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966e2f0-3070-4113-9719-aaf59ad77012 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:45.184207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f72c446-fb8e-4c61-88b2-3e05d8c56d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:45.185021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72c446-fb8e-4c61-88b2-3e05d8c56d4e None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:45.185449 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72c446-fb8e-4c61-88b2-3e05d8c56d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:45.185864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72c446-fb8e-4c61-88b2-3e05d8c56d4e None admin] PATH_INFO: `/v3/groups/a21ac91a329948a2afa5d30d7e107040` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:45.187618 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f72c446-fb8e-4c61-88b2-3e05d8c56d4e None admin] RBAC: Authorizing `identity:get_group(group_id=a21ac91a329948a2afa5d30d7e107040)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:45.189625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f72c446-fb8e-4c61-88b2-3e05d8c56d4e None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:45.192565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 195/392] 10.222.0.74 () {56 vars in 1180 bytes} [Wed Jul 27 16:22:44 2022] GET /identity/v3/groups/a21ac91a329948a2afa5d30d7e107040 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:45.197664 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e5d41-1021-4db0-b3ea-976b69786cda None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:45.245259 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:45.307543 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e5d41-1021-4db0-b3ea-976b69786cda None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:45.343852 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0818bb-16e2-432d-966a-919655809edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:45.344508 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0818bb-16e2-432d-966a-919655809edf None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:45.345080 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0818bb-16e2-432d-966a-919655809edf None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:45.345362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0818bb-16e2-432d-966a-919655809edf None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:45.346524 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0818bb-16e2-432d-966a-919655809edf None admin] RBAC: Authorizing `identity:get_project(project_id=5633253a5837491c995b7f11753765f8)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:45.347959 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0818bb-16e2-432d-966a-919655809edf None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:45.349965 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 198/393] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:45 2022] GET /identity/v3/projects/5633253a5837491c995b7f11753765f8 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:45.357986 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f72c446-fb8e-4c61-88b2-3e05d8c56d4e None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:45.495911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f72c446-fb8e-4c61-88b2-3e05d8c56d4e None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:45.534818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89dabab-5537-426b-aea1-414c686aa483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:45.535440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89dabab-5537-426b-aea1-414c686aa483 None admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:45.535729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89dabab-5537-426b-aea1-414c686aa483 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:45.536008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89dabab-5537-426b-aea1-414c686aa483 None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8/groups/a21ac91a329948a2afa5d30d7e107040/roles/70f3e09bc103410789de8e15c81d9cd3` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:45.538306 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89dabab-5537-426b-aea1-414c686aa483 None admin] RBAC: Authorizing `identity:create_grant(project_id=5633253a5837491c995b7f11753765f8, group_id=a21ac91a329948a2afa5d30d7e107040, role_id=70f3e09bc103410789de8e15c81d9cd3)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:45.539754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89dabab-5537-426b-aea1-414c686aa483 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:45.559658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdee67-71eb-4a4c-aa1f-380489815f11 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:45.559967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdee67-71eb-4a4c-aa1f-380489815f11 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:45.560230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdee67-71eb-4a4c-aa1f-380489815f11 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:45.561668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 199/394] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:45.567175 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644c516-1d01-4c8c-9f3e-e4cf961b8533 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:45.567709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644c516-1d01-4c8c-9f3e-e4cf961b8533 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:45.568273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644c516-1d01-4c8c-9f3e-e4cf961b8533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:45.571493 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 196/395] 10.222.0.74 () {58 vars in 1442 bytes} [Wed Jul 27 16:22:45 2022] PUT /identity/v3/projects/5633253a5837491c995b7f11753765f8/groups/a21ac91a329948a2afa5d30d7e107040/roles/70f3e09bc103410789de8e15c81d9cd3 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:22:45.590928 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-1644c516-1d01-4c8c-9f3e-e4cf961b8533 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:45.592462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-1644c516-1d01-4c8c-9f3e-e4cf961b8533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:45.670961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 200/396] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:45.692300 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89dabab-5537-426b-aea1-414c686aa483 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:45.697316 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:45.809857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89dabab-5537-426b-aea1-414c686aa483 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:45.850848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977264f4-f547-43ec-b91b-52b3ebc5cf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:45.851502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977264f4-f547-43ec-b91b-52b3ebc5cf98 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:45.851782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977264f4-f547-43ec-b91b-52b3ebc5cf98 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:45.852052 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977264f4-f547-43ec-b91b-52b3ebc5cf98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:45.868291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977264f4-f547-43ec-b91b-52b3ebc5cf98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:45.875832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-977264f4-f547-43ec-b91b-52b3ebc5cf98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:45.875832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:45.875832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:45.875832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:45.875832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:45.875832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:45.875832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:45.875832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:45.875832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:45.875832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:45.875832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:45.875832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:45.875832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:45.875832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:45.875832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:45.875832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:45.876782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:45.876782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:45.876782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:45.876782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:45.876782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:45.876782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:45.876782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:45.876782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:45.876782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:45.876782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:45.876782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:45.876782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:45.876782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:45.876782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:45.876782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:45.876782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:45.877604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:45.877604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:45.877604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:45.877604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:45.877604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:45.877604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:45.877604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:45.877604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:45.877604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:45.877604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:45.877604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:45.877604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:45.877604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 197/397] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:45.884731 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644c516-1d01-4c8c-9f3e-e4cf961b8533 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:45.918646 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:45.995198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644c516-1d01-4c8c-9f3e-e4cf961b8533 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:46.029978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffc3bf-689e-428d-bf1c-0504c7078041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:46.030600 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffc3bf-689e-428d-bf1c-0504c7078041 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:46.030879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffc3bf-689e-428d-bf1c-0504c7078041 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:46.031202 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffc3bf-689e-428d-bf1c-0504c7078041 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:46.031920 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ffc3bf-689e-428d-bf1c-0504c7078041 None admin] RBAC: Adding query filter params (name=member) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:46.032292 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ffc3bf-689e-428d-bf1c-0504c7078041 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:46.033697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ffc3bf-689e-428d-bf1c-0504c7078041 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:46.043854 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 201/398] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:46.048443 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977264f4-f547-43ec-b91b-52b3ebc5cf98 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:46.119697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:46.172544 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977264f4-f547-43ec-b91b-52b3ebc5cf98 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:46.210906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c312a64-9bbd-4fa5-93a9-7566b5bceee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:46.211616 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c312a64-9bbd-4fa5-93a9-7566b5bceee3 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:46.211842 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c312a64-9bbd-4fa5-93a9-7566b5bceee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:46.212086 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c312a64-9bbd-4fa5-93a9-7566b5bceee3 None admin] PATH_INFO: `/v3/groups/a21ac91a329948a2afa5d30d7e107040` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:46.213269 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c312a64-9bbd-4fa5-93a9-7566b5bceee3 None admin] RBAC: Authorizing `identity:get_group(group_id=a21ac91a329948a2afa5d30d7e107040)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:46.214718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c312a64-9bbd-4fa5-93a9-7566b5bceee3 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:46.216626 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 198/399] 10.222.0.74 () {56 vars in 1180 bytes} [Wed Jul 27 16:22:46 2022] GET /identity/v3/groups/a21ac91a329948a2afa5d30d7e107040 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:46.222100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffc3bf-689e-428d-bf1c-0504c7078041 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:46.249081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed68242-0ea6-4059-ae96-ebb12017df7a None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:46.249386 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed68242-0ea6-4059-ae96-ebb12017df7a None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:46.249629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed68242-0ea6-4059-ae96-ebb12017df7a None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:46.251246 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 199/400] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:22:46.265225 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62e67e-3e48-4750-b02a-22796b28b546 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:46.265550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62e67e-3e48-4750-b02a-22796b28b546 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:46.265829 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62e67e-3e48-4750-b02a-22796b28b546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:46.293086 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-ee62e67e-3e48-4750-b02a-22796b28b546 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:46.294808 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ee62e67e-3e48-4750-b02a-22796b28b546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:46.401626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:46.405038 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffc3bf-689e-428d-bf1c-0504c7078041 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:46.411531 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 200/401] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:46.434670 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62e67e-3e48-4750-b02a-22796b28b546 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:46.463816 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95dcb25-1a4c-47e1-a092-b26d5974fb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:46.464482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95dcb25-1a4c-47e1-a092-b26d5974fb92 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:46.464768 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95dcb25-1a4c-47e1-a092-b26d5974fb92 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:46.465133 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95dcb25-1a4c-47e1-a092-b26d5974fb92 None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:46.466567 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95dcb25-1a4c-47e1-a092-b26d5974fb92 None admin] RBAC: Authorizing `identity:get_project(project_id=6a66a60a14e247e5bf2142da6f43cc44)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:46.468030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95dcb25-1a4c-47e1-a092-b26d5974fb92 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:46.474003 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 202/402] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:46 2022] GET /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:46.477254 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95dcb25-1a4c-47e1-a092-b26d5974fb92 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:46.517819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:46.603455 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62e67e-3e48-4750-b02a-22796b28b546 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:46.640983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95dcb25-1a4c-47e1-a092-b26d5974fb92 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:46.674074 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602eaee7-ae35-4b53-be8b-e361cae38063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:46.676308 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602eaee7-ae35-4b53-be8b-e361cae38063 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:46.677070 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602eaee7-ae35-4b53-be8b-e361cae38063 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:46.678152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602eaee7-ae35-4b53-be8b-e361cae38063 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:46.701478 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602eaee7-ae35-4b53-be8b-e361cae38063 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:46.702400 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b5a88-896e-4b1d-9c4d-56fc5d0b61c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:46.705131 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b5a88-896e-4b1d-9c4d-56fc5d0b61c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:46.705614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b5a88-896e-4b1d-9c4d-56fc5d0b61c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:46.706072 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b5a88-896e-4b1d-9c4d-56fc5d0b61c4 None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44/groups/a21ac91a329948a2afa5d30d7e107040/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:46.709690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-602eaee7-ae35-4b53-be8b-e361cae38063 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:46.709690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:46.709690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:46.709690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:46.709690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:46.709690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:46.709690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:46.709690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:46.709690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:46.709690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:46.709690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:46.709690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:46.709690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:46.709690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:46.709690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:46.709690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:46.710594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:46.710594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:46.710594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:46.710594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:46.710594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:46.710594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:46.710594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:46.710594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:46.710594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:46.710594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:46.710594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:46.710594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:46.710594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:46.710594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:46.710594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:46.710594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:46.711486 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:46.711486 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:46.711486 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:46.711486 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:46.711486 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:46.711486 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:46.711486 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:46.711486 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:46.711486 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:46.711486 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:46.711486 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:46.711486 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:46.715959 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b5a88-896e-4b1d-9c4d-56fc5d0b61c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a66a60a14e247e5bf2142da6f43cc44, group_id=a21ac91a329948a2afa5d30d7e107040, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:46.721367 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b5a88-896e-4b1d-9c4d-56fc5d0b61c4 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:46.723693 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 201/403] 10.222.0.74 () {56 vars in 1114 bytes} [Wed Jul 27 16:22:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:22:46.727737 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602eaee7-ae35-4b53-be8b-e361cae38063 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:46.745816 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 203/404] 10.222.0.74 () {58 vars in 1442 bytes} [Wed Jul 27 16:22:46 2022] PUT /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44/groups/a21ac91a329948a2afa5d30d7e107040/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:22:46.849861 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602eaee7-ae35-4b53-be8b-e361cae38063 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:46.877433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:46.892165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa29c7-ca2d-4bae-bdef-666ccad0bf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:46.893122 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa29c7-ca2d-4bae-bdef-666ccad0bf55 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:46.893663 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa29c7-ca2d-4bae-bdef-666ccad0bf55 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:46.894156 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa29c7-ca2d-4bae-bdef-666ccad0bf55 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:46.895402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa29c7-ca2d-4bae-bdef-666ccad0bf55 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:46.896190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa29c7-ca2d-4bae-bdef-666ccad0bf55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:46.898409 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa29c7-ca2d-4bae-bdef-666ccad0bf55 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:46.910500 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 202/405] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:22:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:46.916476 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b5a88-896e-4b1d-9c4d-56fc5d0b61c4 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:46.946305 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:47.048692 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b5a88-896e-4b1d-9c4d-56fc5d0b61c4 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:47.085767 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59917f6b-8010-4d65-af00-314dffdada09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:47.086423 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59917f6b-8010-4d65-af00-314dffdada09 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:47.086710 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59917f6b-8010-4d65-af00-314dffdada09 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:47.086987 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59917f6b-8010-4d65-af00-314dffdada09 None admin] PATH_INFO: `/v3/groups/a21ac91a329948a2afa5d30d7e107040` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:47.088138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59917f6b-8010-4d65-af00-314dffdada09 None admin] RBAC: Authorizing `identity:get_group(group_id=a21ac91a329948a2afa5d30d7e107040)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:47.089622 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59917f6b-8010-4d65-af00-314dffdada09 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:47.091574 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 204/406] 10.222.0.74 () {56 vars in 1180 bytes} [Wed Jul 27 16:22:46 2022] GET /identity/v3/groups/a21ac91a329948a2afa5d30d7e107040 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:47.095291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8e5bb-b471-4781-9e47-37a4c6bbfabb None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:47.095584 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8e5bb-b471-4781-9e47-37a4c6bbfabb None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:47.095867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8e5bb-b471-4781-9e47-37a4c6bbfabb None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:47.097671 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 203/407] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:47.099645 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59917f6b-8010-4d65-af00-314dffdada09 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:47.104408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea85cc2-c4f9-43dd-a0bf-3146ec6150b9 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:47.104690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea85cc2-c4f9-43dd-a0bf-3146ec6150b9 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:47.104987 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea85cc2-c4f9-43dd-a0bf-3146ec6150b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:47.131857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-5ea85cc2-c4f9-43dd-a0bf-3146ec6150b9 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:47.134085 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-5ea85cc2-c4f9-43dd-a0bf-3146ec6150b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:47.202856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:47.225323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 204/408] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:47.226777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59917f6b-8010-4d65-af00-314dffdada09 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:47.275947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc41577-00df-44b1-9136-bbdcd2bc8ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:47.276599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc41577-00df-44b1-9136-bbdcd2bc8ddc None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:47.276880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc41577-00df-44b1-9136-bbdcd2bc8ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:47.277178 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc41577-00df-44b1-9136-bbdcd2bc8ddc None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:47.278366 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc41577-00df-44b1-9136-bbdcd2bc8ddc None admin] RBAC: Authorizing `identity:get_project(project_id=6a66a60a14e247e5bf2142da6f43cc44)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:47.279799 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc41577-00df-44b1-9136-bbdcd2bc8ddc None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:47.282752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 205/409] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:47 2022] GET /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:47.287608 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea85cc2-c4f9-43dd-a0bf-3146ec6150b9 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:47.296845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc41577-00df-44b1-9136-bbdcd2bc8ddc None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:47.411926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc41577-00df-44b1-9136-bbdcd2bc8ddc None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:47.452244 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5c1ea-e6af-47e1-9e2c-9feb0368c444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:47.452984 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5c1ea-e6af-47e1-9e2c-9feb0368c444 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:47.453308 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5c1ea-e6af-47e1-9e2c-9feb0368c444 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:47.453482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5c1ea-e6af-47e1-9e2c-9feb0368c444 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:47.466047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c5c1ea-e6af-47e1-9e2c-9feb0368c444 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:47.472007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-b9c5c1ea-e6af-47e1-9e2c-9feb0368c444 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:22:47.472007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:47.472007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:47.472007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:47.472007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:47.472007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:47.472007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:47.472007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:47.472007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:47.472007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:47.472007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:47.472007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:47.472007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:47.472007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:47.472007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:47.472007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:47.472921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:47.472921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:47.472921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:47.472921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:47.472921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:47.472921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:47.472921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:47.472921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:47.472921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:47.472921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:47.472921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:47.472921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:47.472921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:47.472921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:47.472921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:47.472921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:47.473751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:47.473751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:47.473751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:47.473751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:47.473751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:47.473751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:47.473751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:47.473751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:47.473751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:47.473751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:47.473751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:22:47.473751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:47.476792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 206/410] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:22:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:47.479329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5c1ea-e6af-47e1-9e2c-9feb0368c444 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:47.501604 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:47.549768 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea85cc2-c4f9-43dd-a0bf-3146ec6150b9 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:47.573283 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:47.589286 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5c1ea-e6af-47e1-9e2c-9feb0368c444 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:47.626016 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7f9ed-ac45-4831-b543-4438db202be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:47.626645 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7f9ed-ac45-4831-b543-4438db202be4 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:47.626922 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7f9ed-ac45-4831-b543-4438db202be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:47.627102 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc83b1-be40-4552-948c-0b207b0e6fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:47.627274 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7f9ed-ac45-4831-b543-4438db202be4 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:47.627767 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc83b1-be40-4552-948c-0b207b0e6fed None admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:47.627950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b7f9ed-ac45-4831-b543-4438db202be4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:47.628041 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc83b1-be40-4552-948c-0b207b0e6fed None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:47.628237 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b7f9ed-ac45-4831-b543-4438db202be4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:47.628334 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc83b1-be40-4552-948c-0b207b0e6fed None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44/groups/a21ac91a329948a2afa5d30d7e107040/roles/70f3e09bc103410789de8e15c81d9cd3` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:47.629626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b7f9ed-ac45-4831-b543-4438db202be4 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:47.634727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc83b1-be40-4552-948c-0b207b0e6fed None admin] RBAC: Authorizing `identity:create_grant(project_id=6a66a60a14e247e5bf2142da6f43cc44, group_id=a21ac91a329948a2afa5d30d7e107040, role_id=70f3e09bc103410789de8e15c81d9cd3)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:47.636178 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc83b1-be40-4552-948c-0b207b0e6fed None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:47.643333 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 207/411] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:47.644604 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7f9ed-ac45-4831-b543-4438db202be4 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:47.671099 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 205/412] 10.222.0.74 () {58 vars in 1442 bytes} [Wed Jul 27 16:22:47 2022] PUT /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44/groups/a21ac91a329948a2afa5d30d7e107040/roles/70f3e09bc103410789de8e15c81d9cd3 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:22:47.742273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:47.759385 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7f9ed-ac45-4831-b543-4438db202be4 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:47.797765 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5019cf3d-a8e5-4373-9a80-24dbb2918e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:47.798450 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019cf3d-a8e5-4373-9a80-24dbb2918e5e None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:47.798733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019cf3d-a8e5-4373-9a80-24dbb2918e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:47.799009 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019cf3d-a8e5-4373-9a80-24dbb2918e5e None admin] PATH_INFO: `/v3/groups/dad3e0411f804865bb09225e0ee66f55` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:47.800206 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5019cf3d-a8e5-4373-9a80-24dbb2918e5e None admin] RBAC: Authorizing `identity:get_group(group_id=dad3e0411f804865bb09225e0ee66f55)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:47.801702 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5019cf3d-a8e5-4373-9a80-24dbb2918e5e None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:47.808151 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 208/413] 10.222.0.74 () {56 vars in 1180 bytes} [Wed Jul 27 16:22:47 2022] GET /identity/v3/groups/dad3e0411f804865bb09225e0ee66f55 => generated 229 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:47.810044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc83b1-be40-4552-948c-0b207b0e6fed None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:47.861656 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf877b-22e4-4b2f-9b47-9ef2441f10bf None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:47.861988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf877b-22e4-4b2f-9b47-9ef2441f10bf None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:47.862210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf877b-22e4-4b2f-9b47-9ef2441f10bf None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:47.863812 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 209/414] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:22:47.869097 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3814db-4925-4046-bc26-467b14448a3c None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:47.869692 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3814db-4925-4046-bc26-467b14448a3c None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:47.870208 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3814db-4925-4046-bc26-467b14448a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:47.891355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-cd3814db-4925-4046-bc26-467b14448a3c None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:47.893069 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-cd3814db-4925-4046-bc26-467b14448a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:47.978109 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 210/415] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:47.994024 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc83b1-be40-4552-948c-0b207b0e6fed None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:48.002606 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3814db-4925-4046-bc26-467b14448a3c None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:48.033946 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d19c1-7387-4c3c-a728-2d3bc1af753c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:48.034622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d19c1-7387-4c3c-a728-2d3bc1af753c None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:48.034869 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d19c1-7387-4c3c-a728-2d3bc1af753c None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:48.035137 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d19c1-7387-4c3c-a728-2d3bc1af753c None admin] PATH_INFO: `/v3/projects/0d94f0c6f55c441591df5c2cef40a859` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:48.036291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d19c1-7387-4c3c-a728-2d3bc1af753c None admin] RBAC: Authorizing `identity:get_project(project_id=0d94f0c6f55c441591df5c2cef40a859)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:48.037759 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d19c1-7387-4c3c-a728-2d3bc1af753c None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:48.039463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 206/416] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:47 2022] GET /identity/v3/projects/0d94f0c6f55c441591df5c2cef40a859 => generated 344 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:48.045621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d19c1-7387-4c3c-a728-2d3bc1af753c None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:48.092992 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:48.105523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:48.147868 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3814db-4925-4046-bc26-467b14448a3c None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:48.218215 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d19c1-7387-4c3c-a728-2d3bc1af753c None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:48.223521 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca723e2f-8f35-4ab5-aff3-bf2e8bfe6541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:48.224434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca723e2f-8f35-4ab5-aff3-bf2e8bfe6541 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:48.224722 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca723e2f-8f35-4ab5-aff3-bf2e8bfe6541 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:48.225026 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca723e2f-8f35-4ab5-aff3-bf2e8bfe6541 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:48.237472 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca723e2f-8f35-4ab5-aff3-bf2e8bfe6541 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:48.243576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-ca723e2f-8f35-4ab5-aff3-bf2e8bfe6541 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:48.243576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:48.243576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:48.243576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:48.243576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:48.243576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:48.243576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:48.243576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:48.243576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:48.243576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:48.243576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:48.243576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:48.243576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:48.243576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:48.243576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:48.243576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:48.244478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:48.244478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:48.244478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:48.244478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:48.244478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:48.244478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:48.244478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:48.244478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:48.244478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:48.244478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:48.244478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:48.244478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:48.244478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:48.244478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:48.244478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:48.244478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:48.245319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:48.245319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:48.245319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:48.245319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:48.245319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:48.245319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:48.245319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:48.245319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:48.245319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:48.245319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:48.245319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:48.245319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:48.245319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 211/417] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:48.252257 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca723e2f-8f35-4ab5-aff3-bf2e8bfe6541 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:48.295909 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b335475-c9fa-41af-81c5-ab8a171c7e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:48.296602 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b335475-c9fa-41af-81c5-ab8a171c7e7a None admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:48.296903 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b335475-c9fa-41af-81c5-ab8a171c7e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:48.297188 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b335475-c9fa-41af-81c5-ab8a171c7e7a None admin] PATH_INFO: `/v3/projects/0d94f0c6f55c441591df5c2cef40a859/groups/dad3e0411f804865bb09225e0ee66f55/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:48.299774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b335475-c9fa-41af-81c5-ab8a171c7e7a None admin] RBAC: Authorizing `identity:create_grant(project_id=0d94f0c6f55c441591df5c2cef40a859, group_id=dad3e0411f804865bb09225e0ee66f55, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:48.301239 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b335475-c9fa-41af-81c5-ab8a171c7e7a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:48.329780 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 207/418] 10.222.0.74 () {58 vars in 1442 bytes} [Wed Jul 27 16:22:48 2022] PUT /identity/v3/projects/0d94f0c6f55c441591df5c2cef40a859/groups/dad3e0411f804865bb09225e0ee66f55/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:22:48.419413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2510b53-53e8-4953-b4fb-99b4ec8bd29f None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:48.419753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2510b53-53e8-4953-b4fb-99b4ec8bd29f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:48.419973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2510b53-53e8-4953-b4fb-99b4ec8bd29f None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:48.421717 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 208/419] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:22:48.423535 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca723e2f-8f35-4ab5-aff3-bf2e8bfe6541 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:48.427163 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660e2fa-35cb-493f-8600-86ebafb31bac None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:48.427450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660e2fa-35cb-493f-8600-86ebafb31bac None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:48.427729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660e2fa-35cb-493f-8600-86ebafb31bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:48.457211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-c660e2fa-35cb-493f-8600-86ebafb31bac None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:48.458468 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c660e2fa-35cb-493f-8600-86ebafb31bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:48.505612 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605eab83-fe5b-42b4-bb0d-2474671da89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:48.506207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605eab83-fe5b-42b4-bb0d-2474671da89a None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:48.506969 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605eab83-fe5b-42b4-bb0d-2474671da89a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:48.507263 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605eab83-fe5b-42b4-bb0d-2474671da89a None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:48.508284 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605eab83-fe5b-42b4-bb0d-2474671da89a None admin] RBAC: Adding query filter params (name=member) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:48.508605 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605eab83-fe5b-42b4-bb0d-2474671da89a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:48.510366 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605eab83-fe5b-42b4-bb0d-2474671da89a None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:48.519448 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:48.526321 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 212/420] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:48.535140 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605eab83-fe5b-42b4-bb0d-2474671da89a None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:48.552001 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 209/421] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:48.588538 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660e2fa-35cb-493f-8600-86ebafb31bac None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:48.656504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605eab83-fe5b-42b4-bb0d-2474671da89a None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:48.667692 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:48.700693 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8610ad-c0ae-4ddc-97ff-e471b0170909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:48.701488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8610ad-c0ae-4ddc-97ff-e471b0170909 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:48.701872 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8610ad-c0ae-4ddc-97ff-e471b0170909 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:48.702267 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8610ad-c0ae-4ddc-97ff-e471b0170909 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:48.773402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:48.774608 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8610ad-c0ae-4ddc-97ff-e471b0170909 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:48.776333 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8610ad-c0ae-4ddc-97ff-e471b0170909 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:48.846153 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660e2fa-35cb-493f-8600-86ebafb31bac None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:48.919598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 213/422] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 16:22:48.924863 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8610ad-c0ae-4ddc-97ff-e471b0170909 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:48.933107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84245b13-e0ce-44f7-b019-ace8f2d02bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:48.936567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84245b13-e0ce-44f7-b019-ace8f2d02bea None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:48.937125 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84245b13-e0ce-44f7-b019-ace8f2d02bea None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:48.937602 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84245b13-e0ce-44f7-b019-ace8f2d02bea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:48.959890 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84245b13-e0ce-44f7-b019-ace8f2d02bea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:48.966941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-84245b13-e0ce-44f7-b019-ace8f2d02bea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:48.966941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:48.966941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:48.966941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:48.966941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:48.966941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:48.966941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:48.966941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:48.966941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:48.966941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:48.966941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:48.966941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:48.966941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:48.966941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:48.966941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:48.966941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:48.968621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:48.968621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:48.968621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:48.968621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:48.968621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:48.968621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:48.968621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:48.968621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:48.968621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:48.968621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:48.968621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:48.968621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:48.968621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:48.968621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:48.968621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:48.968621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:48.970053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:48.970053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:48.970053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:48.970053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:48.970053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:48.970053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:48.970053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:48.970053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:48.970053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:48.970053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:48.970053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:48.970053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:48.974243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 210/423] 10.222.0.74 () {56 vars in 1114 bytes} [Wed Jul 27 16:22:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:22:48.980760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84245b13-e0ce-44f7-b019-ace8f2d02bea None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:49.087469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:49.098170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84245b13-e0ce-44f7-b019-ace8f2d02bea None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:49.143690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b38d1-7dd4-4797-a574-2da77231fe6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:49.144316 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b38d1-7dd4-4797-a574-2da77231fe6e None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:49.144621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b38d1-7dd4-4797-a574-2da77231fe6e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:49.144863 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b38d1-7dd4-4797-a574-2da77231fe6e None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:49.145991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b38d1-7dd4-4797-a574-2da77231fe6e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:49.146296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b38d1-7dd4-4797-a574-2da77231fe6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:49.147686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b38d1-7dd4-4797-a574-2da77231fe6e None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:49.179406 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 211/424] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:22:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:49.183311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:49.188163 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b38d1-7dd4-4797-a574-2da77231fe6e None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:49.188334 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8610ad-c0ae-4ddc-97ff-e471b0170909 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:49.234301 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94114e4-ae02-45f8-8ca0-3e8487b422c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:49.235021 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94114e4-ae02-45f8-8ca0-3e8487b422c1 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:49.235297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94114e4-ae02-45f8-8ca0-3e8487b422c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:49.235568 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94114e4-ae02-45f8-8ca0-3e8487b422c1 None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:49.239406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94114e4-ae02-45f8-8ca0-3e8487b422c1 None admin] RBAC: Authorizing `identity:get_project(project_id=5633253a5837491c995b7f11753765f8)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:49.240852 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94114e4-ae02-45f8-8ca0-3e8487b422c1 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:49.245182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 214/425] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:48 2022] GET /identity/v3/projects/5633253a5837491c995b7f11753765f8 => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:49.252211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94114e4-ae02-45f8-8ca0-3e8487b422c1 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:49.360962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b38d1-7dd4-4797-a574-2da77231fe6e None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:49.403840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b056b6-4607-47c3-bc52-11b7f2b1d8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:49.404728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b056b6-4607-47c3-bc52-11b7f2b1d8cc None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:49.405976 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b056b6-4607-47c3-bc52-11b7f2b1d8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:49.407775 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b056b6-4607-47c3-bc52-11b7f2b1d8cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:49.460922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:49.468420 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94114e4-ae02-45f8-8ca0-3e8487b422c1 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:49.537446 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b056b6-4607-47c3-bc52-11b7f2b1d8cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:49.539181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b056b6-4607-47c3-bc52-11b7f2b1d8cc None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:49.540200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a21ff-5cb0-4887-9fbf-c376983fdc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:49.540918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a21ff-5cb0-4887-9fbf-c376983fdc30 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:49.541240 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a21ff-5cb0-4887-9fbf-c376983fdc30 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:49.541876 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a21ff-5cb0-4887-9fbf-c376983fdc30 None admin] PATH_INFO: `/v3/groups/a21ac91a329948a2afa5d30d7e107040` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:49.544663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a21ff-5cb0-4887-9fbf-c376983fdc30 None admin] RBAC: Authorizing `identity:get_group(group_id=a21ac91a329948a2afa5d30d7e107040)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:49.546646 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a21ff-5cb0-4887-9fbf-c376983fdc30 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:49.549709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 215/426] 10.222.0.74 () {56 vars in 1180 bytes} [Wed Jul 27 16:22:49 2022] GET /identity/v3/groups/a21ac91a329948a2afa5d30d7e107040 => generated 226 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:49.555896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a21ff-5cb0-4887-9fbf-c376983fdc30 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:49.598096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:49.672020 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 212/427] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:49.673857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e47bb-0879-4e08-98fb-b4701fb081e7 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:49.674298 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e47bb-0879-4e08-98fb-b4701fb081e7 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:49.675070 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e47bb-0879-4e08-98fb-b4701fb081e7 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:49.677186 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 213/428] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:49.678390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e47bb-0879-4e08-98fb-b4701fb081e7 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:49.683324 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a21ff-5cb0-4887-9fbf-c376983fdc30 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:49.725205 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98557bb6-c4c8-4058-a983-fd1914ad3c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:49.725981 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98557bb6-c4c8-4058-a983-fd1914ad3c4a None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:49.726368 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98557bb6-c4c8-4058-a983-fd1914ad3c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:49.726762 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98557bb6-c4c8-4058-a983-fd1914ad3c4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:49.727661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98557bb6-c4c8-4058-a983-fd1914ad3c4a None admin] RBAC: Adding query filter params (group.id=a21ac91a329948a2afa5d30d7e107040, role.id=8de48592893c43789e1d40158943fd52, scope.project.id=5633253a5837491c995b7f11753765f8) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:49.728076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98557bb6-c4c8-4058-a983-fd1914ad3c4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:49.729662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98557bb6-c4c8-4058-a983-fd1914ad3c4a None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:49.738817 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 216/429] 10.222.0.74 () {56 vars in 1376 bytes} [Wed Jul 27 16:22:49 2022] GET /identity/v3/role_assignments?group.id=a21ac91a329948a2afa5d30d7e107040&scope.project.id=5633253a5837491c995b7f11753765f8&role.id=8de48592893c43789e1d40158943fd52 => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:49.746535 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7c935-7d35-4c0b-b45c-41a741b90f5e None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:49.746921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7c935-7d35-4c0b-b45c-41a741b90f5e None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:49.747303 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7c935-7d35-4c0b-b45c-41a741b90f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:49.749513 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:49.790188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-edc7c935-7d35-4c0b-b45c-41a741b90f5e None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:49.792370 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-edc7c935-7d35-4c0b-b45c-41a741b90f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:49.803122 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e47bb-0879-4e08-98fb-b4701fb081e7 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:49.846889 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b7836-9cbe-4219-81bb-d13c35013f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:49.847855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b7836-9cbe-4219-81bb-d13c35013f65 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:49.849246 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b7836-9cbe-4219-81bb-d13c35013f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:49.849734 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b7836-9cbe-4219-81bb-d13c35013f65 None admin] PATH_INFO: `/v3/projects/5633253a5837491c995b7f11753765f8` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:49.853680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b7836-9cbe-4219-81bb-d13c35013f65 None admin] RBAC: Authorizing `identity:get_project(project_id=5633253a5837491c995b7f11753765f8)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:49.858938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b7836-9cbe-4219-81bb-d13c35013f65 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:49.864923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 214/430] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:49 2022] GET /identity/v3/projects/5633253a5837491c995b7f11753765f8 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:49.870838 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b7836-9cbe-4219-81bb-d13c35013f65 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:49.981080 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 217/431] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:50.002586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7c935-7d35-4c0b-b45c-41a741b90f5e None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:50.027791 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b7836-9cbe-4219-81bb-d13c35013f65 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:50.037308 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:50.058217 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:50.076742 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede2451-447a-4548-912d-a5108a89120e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:50.077634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede2451-447a-4548-912d-a5108a89120e None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:50.077921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede2451-447a-4548-912d-a5108a89120e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:50.078199 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede2451-447a-4548-912d-a5108a89120e None admin] PATH_INFO: `/v3/groups/a21ac91a329948a2afa5d30d7e107040` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:50.079763 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede2451-447a-4548-912d-a5108a89120e None admin] RBAC: Authorizing `identity:get_group(group_id=a21ac91a329948a2afa5d30d7e107040)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:50.081735 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede2451-447a-4548-912d-a5108a89120e None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:50.084287 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 215/432] 10.222.0.74 () {56 vars in 1180 bytes} [Wed Jul 27 16:22:49 2022] GET /identity/v3/groups/a21ac91a329948a2afa5d30d7e107040 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:50.094172 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede2451-447a-4548-912d-a5108a89120e None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:50.159056 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7c935-7d35-4c0b-b45c-41a741b90f5e None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:50.206398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c70240-1c19-48e1-b05b-158f323b63ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:50.207325 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c70240-1c19-48e1-b05b-158f323b63ff None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:50.207612 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c70240-1c19-48e1-b05b-158f323b63ff None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:50.207886 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c70240-1c19-48e1-b05b-158f323b63ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:50.224438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c70240-1c19-48e1-b05b-158f323b63ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:50.232475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-d7c70240-1c19-48e1-b05b-158f323b63ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:50.232475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:50.232475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:50.232475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:50.232475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:50.232475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:50.232475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:50.232475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:50.232475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:50.232475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:50.232475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:50.232475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:50.232475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:50.232475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:50.232475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:50.232475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:50.233383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:50.233383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:50.233383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:50.233383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:50.233383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:50.233383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:50.233383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:50.233383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:50.233383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:50.233383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:50.233383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:50.233383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:50.233383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:50.233383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:50.233383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:50.233383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:50.234291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:50.234291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:50.234291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:50.234291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:50.234291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:50.234291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:50.234291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:50.234291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:50.234291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:50.234291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:50.234291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 16:22:50.234291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:50.234291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 218/433] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:22:50.236159 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede2451-447a-4548-912d-a5108a89120e None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:50.239538 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c70240-1c19-48e1-b05b-158f323b63ff None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:50.286067 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686b3e5-d96d-4e5f-858f-e312afc12daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:50.287106 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686b3e5-d96d-4e5f-858f-e312afc12daa None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:50.287602 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686b3e5-d96d-4e5f-858f-e312afc12daa None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:50.288080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686b3e5-d96d-4e5f-858f-e312afc12daa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:50.289117 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686b3e5-d96d-4e5f-858f-e312afc12daa None admin] RBAC: Adding query filter params (group.id=a21ac91a329948a2afa5d30d7e107040, role.id=70f3e09bc103410789de8e15c81d9cd3, scope.project.id=5633253a5837491c995b7f11753765f8) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:50.289644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686b3e5-d96d-4e5f-858f-e312afc12daa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:50.291187 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686b3e5-d96d-4e5f-858f-e312afc12daa None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:50.298970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 216/434] 10.222.0.74 () {56 vars in 1376 bytes} [Wed Jul 27 16:22:50 2022] GET /identity/v3/role_assignments?group.id=a21ac91a329948a2afa5d30d7e107040&scope.project.id=5633253a5837491c995b7f11753765f8&role.id=70f3e09bc103410789de8e15c81d9cd3 => generated 611 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:50.300507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda31b89-94e8-46d1-901e-1689898e8feb None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:50.300793 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda31b89-94e8-46d1-901e-1689898e8feb None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:50.301240 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda31b89-94e8-46d1-901e-1689898e8feb None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:50.303005 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 217/435] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:50.308456 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab040a5-ee04-4ade-8ec2-d2c6257faa68 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:50.308809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab040a5-ee04-4ade-8ec2-d2c6257faa68 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:50.309113 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab040a5-ee04-4ade-8ec2-d2c6257faa68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:50.330884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-3ab040a5-ee04-4ade-8ec2-d2c6257faa68 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:50.332203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-3ab040a5-ee04-4ade-8ec2-d2c6257faa68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:50.352063 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:50.369214 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c70240-1c19-48e1-b05b-158f323b63ff None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:50.416978 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:50.417502 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8acc5bc-0c6d-4da2-b0aa-e9f8959a8c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:50.417802 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acc5bc-0c6d-4da2-b0aa-e9f8959a8c9f None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:50.418163 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acc5bc-0c6d-4da2-b0aa-e9f8959a8c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:50.419017 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acc5bc-0c6d-4da2-b0aa-e9f8959a8c9f None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:50.419809 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8acc5bc-0c6d-4da2-b0aa-e9f8959a8c9f None admin] RBAC: Adding query filter params (name=member) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:50.420114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8acc5bc-0c6d-4da2-b0aa-e9f8959a8c9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:50.421712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8acc5bc-0c6d-4da2-b0aa-e9f8959a8c9f None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:50.433896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 219/436] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:22:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:50.439427 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8acc5bc-0c6d-4da2-b0aa-e9f8959a8c9f None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:50.439961 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 218/437] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:50.470252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab040a5-ee04-4ade-8ec2-d2c6257faa68 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:50.572834 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8acc5bc-0c6d-4da2-b0aa-e9f8959a8c9f None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:50.600767 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab040a5-ee04-4ade-8ec2-d2c6257faa68 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:50.618348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ecbd68-5a9d-4cb5-a84b-98b81aff0581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:50.618348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecbd68-5a9d-4cb5-a84b-98b81aff0581 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:50.618601 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecbd68-5a9d-4cb5-a84b-98b81aff0581 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:50.618816 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecbd68-5a9d-4cb5-a84b-98b81aff0581 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:50.650472 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1db53-b195-404e-86de-150d72e4ddcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:50.651145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:50.651565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1db53-b195-404e-86de-150d72e4ddcb None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:50.651856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1db53-b195-404e-86de-150d72e4ddcb None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:50.652134 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1db53-b195-404e-86de-150d72e4ddcb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:50.668573 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd1db53-b195-404e-86de-150d72e4ddcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:50.673579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-3dd1db53-b195-404e-86de-150d72e4ddcb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:50.673579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:50.673579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:50.673579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:50.673579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:50.673579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:50.673579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:50.673579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:50.673579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:50.673579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:50.673579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:50.673579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:50.673579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:50.673579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:50.673579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:50.673579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:50.674471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:50.674471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:50.674471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:50.674471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:50.674471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:50.674471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:50.674471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:50.674471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:50.674471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:50.674471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:50.674471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:50.674471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:50.674471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:50.674471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:50.674471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:50.674471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:50.675289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:50.675289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:50.675289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:50.675289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:50.675289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:50.675289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:50.675289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:50.675289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:50.675289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:50.675289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:50.675289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:50.675289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:50.676928 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 219/438] 10.222.0.74 () {56 vars in 1114 bytes} [Wed Jul 27 16:22:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:22:50.682568 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1db53-b195-404e-86de-150d72e4ddcb None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:50.722438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ecbd68-5a9d-4cb5-a84b-98b81aff0581 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:50.724040 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ecbd68-5a9d-4cb5-a84b-98b81aff0581 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:50.741827 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:50.802800 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1db53-b195-404e-86de-150d72e4ddcb None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:50.835475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 220/439] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:50.836642 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90659ccd-c27d-4570-b946-53d0db7f3b68 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:50.836856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0760e8-40f9-41a5-a814-e1b9237ac36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:50.836995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90659ccd-c27d-4570-b946-53d0db7f3b68 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:50.837209 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90659ccd-c27d-4570-b946-53d0db7f3b68 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:50.837491 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0760e8-40f9-41a5-a814-e1b9237ac36f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:50.837779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0760e8-40f9-41a5-a814-e1b9237ac36f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:50.838058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0760e8-40f9-41a5-a814-e1b9237ac36f None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:50.838580 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 221/440] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:50.838838 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0760e8-40f9-41a5-a814-e1b9237ac36f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:50.839151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0760e8-40f9-41a5-a814-e1b9237ac36f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:50.841270 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0760e8-40f9-41a5-a814-e1b9237ac36f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:50.843266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90659ccd-c27d-4570-b946-53d0db7f3b68 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:50.849770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 220/441] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:22:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:50.851016 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9838072-5e58-4200-a816-6a0b87cf5591 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:50.851544 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9838072-5e58-4200-a816-6a0b87cf5591 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:50.851673 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9838072-5e58-4200-a816-6a0b87cf5591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:50.868757 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-b9838072-5e58-4200-a816-6a0b87cf5591 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:50.869986 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b9838072-5e58-4200-a816-6a0b87cf5591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:50.896135 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:50.935495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 221/442] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:50.936326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9838072-5e58-4200-a816-6a0b87cf5591 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:50.952336 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90659ccd-c27d-4570-b946-53d0db7f3b68 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:50.971046 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:50.986465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be8140-1467-421a-bf1f-772d69db002b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:50.987104 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be8140-1467-421a-bf1f-772d69db002b None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:50.987378 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be8140-1467-421a-bf1f-772d69db002b None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:50.987643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be8140-1467-421a-bf1f-772d69db002b None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:50.988768 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be8140-1467-421a-bf1f-772d69db002b None admin] RBAC: Authorizing `identity:get_project(project_id=6a66a60a14e247e5bf2142da6f43cc44)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:50.990231 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be8140-1467-421a-bf1f-772d69db002b None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:50.991854 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 222/443] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:50 2022] GET /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:50.993730 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be8140-1467-421a-bf1f-772d69db002b None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:51.036806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9838072-5e58-4200-a816-6a0b87cf5591 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:51.070533 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b47d8-1f08-4d7d-b412-db3db8efd07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:51.071154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b47d8-1f08-4d7d-b412-db3db8efd07e None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:51.071439 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b47d8-1f08-4d7d-b412-db3db8efd07e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:51.071717 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b47d8-1f08-4d7d-b412-db3db8efd07e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:51.089482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be8140-1467-421a-bf1f-772d69db002b None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:51.109570 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:51.124466 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe4038-4afc-45d5-9393-5f48979d3987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:51.125128 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe4038-4afc-45d5-9393-5f48979d3987 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:51.125421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe4038-4afc-45d5-9393-5f48979d3987 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:51.125694 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe4038-4afc-45d5-9393-5f48979d3987 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:51.135511 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe4038-4afc-45d5-9393-5f48979d3987 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:51.138118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b47d8-1f08-4d7d-b412-db3db8efd07e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:51.139600 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b47d8-1f08-4d7d-b412-db3db8efd07e None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:51.140963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-effe4038-4afc-45d5-9393-5f48979d3987 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:22:51.140963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:51.140963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:51.140963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:51.140963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:51.140963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:51.140963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:51.140963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:51.140963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:51.140963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:51.140963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:51.140963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:51.140963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:51.140963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:51.140963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:51.140963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:51.141978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:51.141978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:51.141978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:51.141978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:51.141978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:51.141978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:51.141978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:51.141978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:51.141978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:51.141978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:51.141978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:51.141978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:51.141978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:51.141978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:51.141978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:51.141978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:51.142773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:51.142773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:51.142773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:51.142773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:51.142773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:51.142773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:51.142773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:51.142773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:51.142773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:51.142773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:51.142773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:22:51.142773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:51.142773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 223/444] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:22:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:51.143416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe4038-4afc-45d5-9393-5f48979d3987 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:51.176982 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:51.240597 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 222/445] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:51.241377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b47d8-1f08-4d7d-b412-db3db8efd07e None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:51.247516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe4038-4afc-45d5-9393-5f48979d3987 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:51.281914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aec95b-1c4f-4eba-8c81-d7e7489adaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:51.282546 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aec95b-1c4f-4eba-8c81-d7e7489adaa0 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:51.282828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aec95b-1c4f-4eba-8c81-d7e7489adaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:51.283098 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aec95b-1c4f-4eba-8c81-d7e7489adaa0 None admin] PATH_INFO: `/v3/groups/a21ac91a329948a2afa5d30d7e107040` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:51.284238 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aec95b-1c4f-4eba-8c81-d7e7489adaa0 None admin] RBAC: Authorizing `identity:get_group(group_id=a21ac91a329948a2afa5d30d7e107040)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:51.285682 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aec95b-1c4f-4eba-8c81-d7e7489adaa0 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:51.287251 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 224/446] 10.222.0.74 () {56 vars in 1180 bytes} [Wed Jul 27 16:22:51 2022] GET /identity/v3/groups/a21ac91a329948a2afa5d30d7e107040 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:51.287990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aec95b-1c4f-4eba-8c81-d7e7489adaa0 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:51.338593 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:51.338875 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b47d8-1f08-4d7d-b412-db3db8efd07e None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:51.372099 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fcd17-9861-4ec7-865d-b6a01aa828b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:51.372672 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fcd17-9861-4ec7-865d-b6a01aa828b3 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:51.372973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fcd17-9861-4ec7-865d-b6a01aa828b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:51.373249 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fcd17-9861-4ec7-865d-b6a01aa828b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:51.373959 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9fcd17-9861-4ec7-865d-b6a01aa828b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:51.374259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9fcd17-9861-4ec7-865d-b6a01aa828b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:51.375627 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9fcd17-9861-4ec7-865d-b6a01aa828b3 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:51.377761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:51.384615 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 223/447] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:22:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:51.385310 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fcd17-9861-4ec7-865d-b6a01aa828b3 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:51.390348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aec95b-1c4f-4eba-8c81-d7e7489adaa0 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:51.424529 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb87c7-aeb3-4b1b-a9a2-f4e2820ce9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:51.425214 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb87c7-aeb3-4b1b-a9a2-f4e2820ce9c8 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:51.425500 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb87c7-aeb3-4b1b-a9a2-f4e2820ce9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:51.425777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb87c7-aeb3-4b1b-a9a2-f4e2820ce9c8 None admin] PATH_INFO: `/v3/projects/6a66a60a14e247e5bf2142da6f43cc44` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:51.426932 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb87c7-aeb3-4b1b-a9a2-f4e2820ce9c8 None admin] RBAC: Authorizing `identity:get_project(project_id=6a66a60a14e247e5bf2142da6f43cc44)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:51.428360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb87c7-aeb3-4b1b-a9a2-f4e2820ce9c8 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:51.430043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 225/448] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:51 2022] GET /identity/v3/projects/6a66a60a14e247e5bf2142da6f43cc44 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:51.430792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb87c7-aeb3-4b1b-a9a2-f4e2820ce9c8 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:51.488562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fcd17-9861-4ec7-865d-b6a01aa828b3 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:51.520807 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424af10-a3d5-4cf7-b21f-d758e4b8d5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:51.521521 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424af10-a3d5-4cf7-b21f-d758e4b8d5ab None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:51.521804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424af10-a3d5-4cf7-b21f-d758e4b8d5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:51.522079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424af10-a3d5-4cf7-b21f-d758e4b8d5ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:51.522859 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424af10-a3d5-4cf7-b21f-d758e4b8d5ab None admin] RBAC: Adding query filter params (group.id=a21ac91a329948a2afa5d30d7e107040, role.id=8de48592893c43789e1d40158943fd52, scope.project.id=6a66a60a14e247e5bf2142da6f43cc44) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:51.523171 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424af10-a3d5-4cf7-b21f-d758e4b8d5ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:51.524601 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424af10-a3d5-4cf7-b21f-d758e4b8d5ab None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:51.529378 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb87c7-aeb3-4b1b-a9a2-f4e2820ce9c8 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:51.530326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 224/449] 10.222.0.74 () {56 vars in 1376 bytes} [Wed Jul 27 16:22:51 2022] GET /identity/v3/role_assignments?group.id=a21ac91a329948a2afa5d30d7e107040&scope.project.id=6a66a60a14e247e5bf2142da6f43cc44&role.id=8de48592893c43789e1d40158943fd52 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:51.531101 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424af10-a3d5-4cf7-b21f-d758e4b8d5ab None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:51.549808 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:51.571805 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9377936-930e-42a8-917e-e9773f9537cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:51.572564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9377936-930e-42a8-917e-e9773f9537cd None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:51.572848 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9377936-930e-42a8-917e-e9773f9537cd None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:51.573443 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9377936-930e-42a8-917e-e9773f9537cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:51.609328 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:51.653891 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9377936-930e-42a8-917e-e9773f9537cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:51.655277 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9377936-930e-42a8-917e-e9773f9537cd None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:51.656159 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424af10-a3d5-4cf7-b21f-d758e4b8d5ab None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:51.697571 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede7025-2c8d-4094-a23c-c5cffa7e2d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:51.698225 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede7025-2c8d-4094-a23c-c5cffa7e2d55 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:51.698507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede7025-2c8d-4094-a23c-c5cffa7e2d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:51.698781 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede7025-2c8d-4094-a23c-c5cffa7e2d55 None admin] PATH_INFO: `/v3/groups/a21ac91a329948a2afa5d30d7e107040` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:51.703050 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede7025-2c8d-4094-a23c-c5cffa7e2d55 None admin] RBAC: Authorizing `identity:get_group(group_id=a21ac91a329948a2afa5d30d7e107040)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:51.703050 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede7025-2c8d-4094-a23c-c5cffa7e2d55 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:51.703578 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 225/450] 10.222.0.74 () {56 vars in 1180 bytes} [Wed Jul 27 16:22:51 2022] GET /identity/v3/groups/a21ac91a329948a2afa5d30d7e107040 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:51.709416 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede7025-2c8d-4094-a23c-c5cffa7e2d55 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:51.778304 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 226/451] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:51.783808 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9377936-930e-42a8-917e-e9773f9537cd None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:51.806466 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:51.834440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede7025-2c8d-4094-a23c-c5cffa7e2d55 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:51.869684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:51.878644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427380dd-4473-4941-80b2-01b9f6b54ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:51.879272 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427380dd-4473-4941-80b2-01b9f6b54ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:51.879556 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427380dd-4473-4941-80b2-01b9f6b54ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:51.879838 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427380dd-4473-4941-80b2-01b9f6b54ca0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:51.880637 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427380dd-4473-4941-80b2-01b9f6b54ca0 None admin] RBAC: Adding query filter params (group.id=a21ac91a329948a2afa5d30d7e107040, role.id=70f3e09bc103410789de8e15c81d9cd3, scope.project.id=6a66a60a14e247e5bf2142da6f43cc44) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:51.880975 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427380dd-4473-4941-80b2-01b9f6b54ca0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:51.882423 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427380dd-4473-4941-80b2-01b9f6b54ca0 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:51.889098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 226/452] 10.222.0.74 () {56 vars in 1376 bytes} [Wed Jul 27 16:22:51 2022] GET /identity/v3/role_assignments?group.id=a21ac91a329948a2afa5d30d7e107040&scope.project.id=6a66a60a14e247e5bf2142da6f43cc44&role.id=70f3e09bc103410789de8e15c81d9cd3 => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:51.912180 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9377936-930e-42a8-917e-e9773f9537cd None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:51.949036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc0440-9145-46e2-9bff-2d402744241a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:51.949746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc0440-9145-46e2-9bff-2d402744241a None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:51.950030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc0440-9145-46e2-9bff-2d402744241a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:51.950563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc0440-9145-46e2-9bff-2d402744241a None admin] PATH_INFO: `/v3/projects/0d94f0c6f55c441591df5c2cef40a859` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:51.951794 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc0440-9145-46e2-9bff-2d402744241a None admin] RBAC: Authorizing `identity:get_project(project_id=0d94f0c6f55c441591df5c2cef40a859)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:51.953313 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc0440-9145-46e2-9bff-2d402744241a None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:51.955044 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 227/453] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:22:51 2022] GET /identity/v3/projects/0d94f0c6f55c441591df5c2cef40a859 => generated 344 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:51.960841 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427380dd-4473-4941-80b2-01b9f6b54ca0 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:52.068570 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427380dd-4473-4941-80b2-01b9f6b54ca0 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:52.102821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57199d29-f5d2-4b79-bce3-2e2d7e7fd935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:52.103737 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57199d29-f5d2-4b79-bce3-2e2d7e7fd935 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:52.104014 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57199d29-f5d2-4b79-bce3-2e2d7e7fd935 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:52.104287 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57199d29-f5d2-4b79-bce3-2e2d7e7fd935 None admin] PATH_INFO: `/v3/groups/dad3e0411f804865bb09225e0ee66f55` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:52.106144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57199d29-f5d2-4b79-bce3-2e2d7e7fd935 None admin] RBAC: Authorizing `identity:get_group(group_id=dad3e0411f804865bb09225e0ee66f55)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:52.107705 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57199d29-f5d2-4b79-bce3-2e2d7e7fd935 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:52.109339 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 227/454] 10.222.0.74 () {56 vars in 1180 bytes} [Wed Jul 27 16:22:51 2022] GET /identity/v3/groups/dad3e0411f804865bb09225e0ee66f55 => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:52.115354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc0440-9145-46e2-9bff-2d402744241a None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:52.206476 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:52.225302 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc0440-9145-46e2-9bff-2d402744241a None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:52.259096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b399498-0447-4001-a800-4f1ffb17be56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:52.259716 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b399498-0447-4001-a800-4f1ffb17be56 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:52.259994 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b399498-0447-4001-a800-4f1ffb17be56 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:52.260265 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b399498-0447-4001-a800-4f1ffb17be56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:52.261073 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b399498-0447-4001-a800-4f1ffb17be56 None admin] RBAC: Adding query filter params (group.id=dad3e0411f804865bb09225e0ee66f55, role.id=ca37e43082fa40c6a88b12680a0ca94b, scope.project.id=0d94f0c6f55c441591df5c2cef40a859) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:52.261381 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b399498-0447-4001-a800-4f1ffb17be56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:52.263329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b399498-0447-4001-a800-4f1ffb17be56 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:52.268830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 228/455] 10.222.0.74 () {56 vars in 1376 bytes} [Wed Jul 27 16:22:52 2022] GET /identity/v3/role_assignments?group.id=dad3e0411f804865bb09225e0ee66f55&scope.project.id=0d94f0c6f55c441591df5c2cef40a859&role.id=ca37e43082fa40c6a88b12680a0ca94b => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:54.693468 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75eaedb-caeb-4092-82e5-b39e5269ce49 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:54.694019 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75eaedb-caeb-4092-82e5-b39e5269ce49 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:54.694019 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75eaedb-caeb-4092-82e5-b39e5269ce49 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:54.695592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 228/456] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:54.701681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f601e-b7c0-4284-aa61-33c976f98199 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:54.702484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f601e-b7c0-4284-aa61-33c976f98199 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:54.703059 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f601e-b7c0-4284-aa61-33c976f98199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:54.729829 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-b07f601e-b7c0-4284-aa61-33c976f98199 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:54.731845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b07f601e-b7c0-4284-aa61-33c976f98199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:54.826473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 229/457] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:54.859532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75eaedb-caeb-4092-82e5-b39e5269ce49 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:54.944862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:55.014616 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75eaedb-caeb-4092-82e5-b39e5269ce49 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:55.042522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd78ac-3596-46a9-b27b-183574f400b4 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:55.043021 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd78ac-3596-46a9-b27b-183574f400b4 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:55.043397 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd78ac-3596-46a9-b27b-183574f400b4 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:55.045325 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 230/458] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:55.052065 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a825c-0eda-48ac-9719-e18c4edfce31 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:55.052630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a825c-0eda-48ac-9719-e18c4edfce31 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:55.053142 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a825c-0eda-48ac-9719-e18c4edfce31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:55.075088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222bcd2-4630-4a12-9aa4-d20c12ee6102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:55.075999 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222bcd2-4630-4a12-9aa4-d20c12ee6102 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:55.076281 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222bcd2-4630-4a12-9aa4-d20c12ee6102 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:55.076560 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222bcd2-4630-4a12-9aa4-d20c12ee6102 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:55.092088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-be4a825c-0eda-48ac-9719-e18c4edfce31 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:55.093310 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-be4a825c-0eda-48ac-9719-e18c4edfce31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:55.097366 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b222bcd2-4630-4a12-9aa4-d20c12ee6102 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:55.098852 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b222bcd2-4630-4a12-9aa4-d20c12ee6102 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:55.126567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-b222bcd2-4630-4a12-9aa4-d20c12ee6102 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:22:55.126567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:55.126567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:55.126567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:55.126567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:55.126567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:55.126567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:55.126567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:55.126567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:55.126567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:55.126567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:55.126567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:55.126567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:22:55.126567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:22:55.126567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:22:55.127474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:22:55.127474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:55.127474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:55.127474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:55.127474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:55.127474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:55.127474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:55.127474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:55.127474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:55.127474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:55.127474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:55.127474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:55.127474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:55.127474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:55.127474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:55.127474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:55.127474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:55.128296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:22:55.128296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:22:55.128296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:22:55.128296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:55.131009 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 229/459] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:22:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:55.136474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222bcd2-4630-4a12-9aa4-d20c12ee6102 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:55.242161 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 231/460] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:55.248911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222bcd2-4630-4a12-9aa4-d20c12ee6102 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:55.249045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72af8e-e18b-4f2d-aca2-3670ab2e8511 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:55.249045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72af8e-e18b-4f2d-aca2-3670ab2e8511 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:55.249045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72af8e-e18b-4f2d-aca2-3670ab2e8511 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:55.249045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 232/461] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:55.251723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd04f0-3159-496f-be0d-d49f6b6ae6db None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:55.252013 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd04f0-3159-496f-be0d-d49f6b6ae6db None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:55.252289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd04f0-3159-496f-be0d-d49f6b6ae6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:55.271376 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-31dd04f0-3159-496f-be0d-d49f6b6ae6db None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:55.272492 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-31dd04f0-3159-496f-be0d-d49f6b6ae6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:55.277190 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:55.283196 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:55.307194 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db45809-1476-4a4c-8340-bc2b0fe2135a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:55.307951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db45809-1476-4a4c-8340-bc2b0fe2135a None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:55.308382 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db45809-1476-4a4c-8340-bc2b0fe2135a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:55.309378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db45809-1476-4a4c-8340-bc2b0fe2135a None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:55.310171 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db45809-1476-4a4c-8340-bc2b0fe2135a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:55.310598 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db45809-1476-4a4c-8340-bc2b0fe2135a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:55.312064 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db45809-1476-4a4c-8340-bc2b0fe2135a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:55.330599 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 230/462] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:55.331844 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db45809-1476-4a4c-8340-bc2b0fe2135a None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:55.360751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 233/463] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:55.362535 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd04f0-3159-496f-be0d-d49f6b6ae6db None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:55.467562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db45809-1476-4a4c-8340-bc2b0fe2135a None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:55.485164 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd04f0-3159-496f-be0d-d49f6b6ae6db None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:55.506815 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91172283-bc32-4288-b19a-5fff4f13a71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:55.507805 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91172283-bc32-4288-b19a-5fff4f13a71a None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:55.508098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91172283-bc32-4288-b19a-5fff4f13a71a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:55.508377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91172283-bc32-4288-b19a-5fff4f13a71a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:55.518647 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91172283-bc32-4288-b19a-5fff4f13a71a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:55.520041 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91172283-bc32-4288-b19a-5fff4f13a71a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:55.522093 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee4eae-a74c-45b2-9e8d-b8acbf0bce52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:55.522838 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee4eae-a74c-45b2-9e8d-b8acbf0bce52 None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:55.523123 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee4eae-a74c-45b2-9e8d-b8acbf0bce52 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:55.523399 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee4eae-a74c-45b2-9e8d-b8acbf0bce52 None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:55.524076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee4eae-a74c-45b2-9e8d-b8acbf0bce52 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:55.525539 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee4eae-a74c-45b2-9e8d-b8acbf0bce52 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:55.535304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-91172283-bc32-4288-b19a-5fff4f13a71a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:22:55.535304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:55.535304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:55.535304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:55.535304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:55.535304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:55.535304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:55.535304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:55.535304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:55.535304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:55.535304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:55.535304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:55.535304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:22:55.535304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:22:55.535304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:22:55.536158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:22:55.536158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:55.536158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:55.536158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:55.536158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:55.536158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:55.536158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:55.536158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:55.536158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:55.536158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:55.536158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:55.536158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:55.536158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:55.536158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:55.536158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:55.536158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:55.536158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:55.537039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:22:55.537039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:22:55.537039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:22:55.537039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:55.537039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 231/464] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:55.537295 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91172283-bc32-4288-b19a-5fff4f13a71a None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:55.538804 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:55.554577 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 234/465] 10.222.0.74 () {60 vars in 1132 bytes} [Wed Jul 27 16:22:55 2022] POST /identity/v3/users => generated 246 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:22:55.556202 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453a347-f865-4266-8f98-650cebd90906 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:55.556515 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453a347-f865-4266-8f98-650cebd90906 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:55.556795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453a347-f865-4266-8f98-650cebd90906 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:55.558128 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 235/466] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:55.558904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453a347-f865-4266-8f98-650cebd90906 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:55.586671 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:55.679905 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91172283-bc32-4288-b19a-5fff4f13a71a None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:55.711849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453a347-f865-4266-8f98-650cebd90906 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:55.731443 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e19315-971a-4a77-8fd5-6e24a2aec9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:55.732647 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e19315-971a-4a77-8fd5-6e24a2aec9f7 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:55.733149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e19315-971a-4a77-8fd5-6e24a2aec9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:55.733429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e19315-971a-4a77-8fd5-6e24a2aec9f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:55.745083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e19315-971a-4a77-8fd5-6e24a2aec9f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:55.747163 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e19315-971a-4a77-8fd5-6e24a2aec9f7 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:55.759369 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c5321c-5862-40f4-b3d8-7ed72296da1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:55.759369 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5321c-5862-40f4-b3d8-7ed72296da1a None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:55.759745 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5321c-5862-40f4-b3d8-7ed72296da1a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:55.759907 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5321c-5862-40f4-b3d8-7ed72296da1a None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:55.760611 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c5321c-5862-40f4-b3d8-7ed72296da1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:55.760954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c5321c-5862-40f4-b3d8-7ed72296da1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:55.762486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c5321c-5862-40f4-b3d8-7ed72296da1a None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:55.766990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-a2e19315-971a-4a77-8fd5-6e24a2aec9f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:22:55.766990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:55.766990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:55.766990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:55.766990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:55.766990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:55.766990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:55.766990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:55.766990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:55.766990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:55.766990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:55.766990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:55.766990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:22:55.766990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:22:55.766990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:22:55.767984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:22:55.767984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:55.767984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:55.767984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:55.767984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:55.767984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:55.767984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:55.767984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:55.767984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:55.767984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:55.767984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:55.767984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:55.767984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:55.767984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:55.767984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:55.767984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:55.767984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:55.769105 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:22:55.769105 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:22:55.769105 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:22:55.769105 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:55.771457 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 232/467] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:55.778411 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444e27b-5376-4f9e-9b01-956cff25ca1f None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:55.778695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444e27b-5376-4f9e-9b01-956cff25ca1f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:55.779622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444e27b-5376-4f9e-9b01-956cff25ca1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:55.783028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 236/468] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:55.783794 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c5321c-5862-40f4-b3d8-7ed72296da1a None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:55.804168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-3444e27b-5376-4f9e-9b01-956cff25ca1f None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:55.805660 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-3444e27b-5376-4f9e-9b01-956cff25ca1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:55.856795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:55.886479 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 233/469] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:55.892910 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444e27b-5376-4f9e-9b01-956cff25ca1f None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:55.911816 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:55.927209 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c5321c-5862-40f4-b3d8-7ed72296da1a None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:55.974107 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601616f8-a0fe-4f55-89fd-cd07cd0e5d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:55.975309 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601616f8-a0fe-4f55-89fd-cd07cd0e5d88 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:55.975792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601616f8-a0fe-4f55-89fd-cd07cd0e5d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:55.976876 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601616f8-a0fe-4f55-89fd-cd07cd0e5d88 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:55.978094 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601616f8-a0fe-4f55-89fd-cd07cd0e5d88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:55.978600 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601616f8-a0fe-4f55-89fd-cd07cd0e5d88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:55.981594 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601616f8-a0fe-4f55-89fd-cd07cd0e5d88 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:56.002176 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 237/470] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:56.002176 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601616f8-a0fe-4f55-89fd-cd07cd0e5d88 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:56.019830 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444e27b-5376-4f9e-9b01-956cff25ca1f None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:56.055752 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862ac83-752b-4333-9367-04a86349c199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:56.056399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862ac83-752b-4333-9367-04a86349c199 None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:56.056651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862ac83-752b-4333-9367-04a86349c199 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:56.056949 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862ac83-752b-4333-9367-04a86349c199 None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:56.057626 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862ac83-752b-4333-9367-04a86349c199 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:56.059206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862ac83-752b-4333-9367-04a86349c199 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:56.087876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 234/471] 10.222.0.74 () {60 vars in 1132 bytes} [Wed Jul 27 16:22:55 2022] POST /identity/v3/users => generated 248 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:22:56.087876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac6013-086b-4a8e-97ff-c978e5ad0334 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:56.087876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac6013-086b-4a8e-97ff-c978e5ad0334 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:56.087876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac6013-086b-4a8e-97ff-c978e5ad0334 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:56.087876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 235/472] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:56.087876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac6013-086b-4a8e-97ff-c978e5ad0334 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:56.117114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601616f8-a0fe-4f55-89fd-cd07cd0e5d88 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:56.134002 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:56.167519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f4364-84c2-41ff-8682-99418498e737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:56.173590 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f4364-84c2-41ff-8682-99418498e737 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:56.173590 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f4364-84c2-41ff-8682-99418498e737 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:56.173590 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f4364-84c2-41ff-8682-99418498e737 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:56.179659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:56.192510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f4364-84c2-41ff-8682-99418498e737 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:56.194446 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f4364-84c2-41ff-8682-99418498e737 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:56.220814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-411f4364-84c2-41ff-8682-99418498e737 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:22:56.220814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:56.220814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:56.220814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:56.220814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:56.220814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:56.220814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:56.220814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:56.220814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:56.220814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:56.220814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:56.220814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:56.220814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:22:56.220814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:22:56.220814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:22:56.221759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:22:56.221759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:56.221759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:56.221759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:56.221759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:56.221759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:56.221759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:56.221759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:56.221759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:56.221759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:56.221759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:56.221759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:56.221759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:56.221759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:56.221759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:56.221759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:56.221759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:56.222583 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:22:56.222583 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:22:56.222583 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:22:56.222583 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:56.224853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 238/473] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:56.226573 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ed71e-857c-4084-8834-709d2608b2e5 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:56.227065 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ed71e-857c-4084-8834-709d2608b2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:56.227537 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ed71e-857c-4084-8834-709d2608b2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:56.233425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac6013-086b-4a8e-97ff-c978e5ad0334 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:56.250410 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-b49ed71e-857c-4084-8834-709d2608b2e5 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:56.252086 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b49ed71e-857c-4084-8834-709d2608b2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:56.280322 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc98c4-31b1-411e-9eee-a6db0bb90076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:56.281209 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc98c4-31b1-411e-9eee-a6db0bb90076 None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:56.281656 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc98c4-31b1-411e-9eee-a6db0bb90076 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:56.282074 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc98c4-31b1-411e-9eee-a6db0bb90076 None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:56.282916 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc98c4-31b1-411e-9eee-a6db0bb90076 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:56.284390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc98c4-31b1-411e-9eee-a6db0bb90076 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:56.310899 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 236/474] 10.222.0.74 () {60 vars in 1132 bytes} [Wed Jul 27 16:22:56 2022] POST /identity/v3/users => generated 248 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:22:56.311903 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc98c4-31b1-411e-9eee-a6db0bb90076 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:56.344844 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 239/475] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:56.370620 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ed71e-857c-4084-8834-709d2608b2e5 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:56.455921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc98c4-31b1-411e-9eee-a6db0bb90076 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:56.481792 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:56.514227 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6b024-21f2-49e2-8399-9cf099c425ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:56.515135 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6b024-21f2-49e2-8399-9cf099c425ce None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:56.515413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6b024-21f2-49e2-8399-9cf099c425ce None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:56.515683 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6b024-21f2-49e2-8399-9cf099c425ce None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:56.516377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6b024-21f2-49e2-8399-9cf099c425ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:56.516678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6b024-21f2-49e2-8399-9cf099c425ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:56.518305 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6b024-21f2-49e2-8399-9cf099c425ce None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:56.529712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:56.536960 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 237/476] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:56.537104 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ed71e-857c-4084-8834-709d2608b2e5 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:56.540510 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6b024-21f2-49e2-8399-9cf099c425ce None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:56.577040 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fbace-3c1c-4fc4-8004-eadbdf362c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:56.577775 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fbace-3c1c-4fc4-8004-eadbdf362c43 None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:56.578129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fbace-3c1c-4fc4-8004-eadbdf362c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:56.578474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fbace-3c1c-4fc4-8004-eadbdf362c43 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:56.579535 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560fbace-3c1c-4fc4-8004-eadbdf362c43 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:56.581771 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560fbace-3c1c-4fc4-8004-eadbdf362c43 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:56.596775 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.sql.core [None req-560fbace-3c1c-4fc4-8004-eadbdf362c43 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 16:22:56.596775 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 27 16:22:56.596775 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [parameters: {'id': '6c8f92dc9b3d420ca0637cfc49ba0e05', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 27 16:22:56.596775 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80109) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 16:22:56.619746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-560fbace-3c1c-4fc4-8004-eadbdf362c43 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 27 16:22:56.619746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:56.619746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 16:22:56.619746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 16:22:56.619746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 16:22:56.619746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 16:22:56.619746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 16:22:56.619746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 16:22:56.619746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 16:22:56.619746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application conn.query(q) Jul 27 16:22:56.619746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 16:22:56.619746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 16:22:56.619746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 16:22:56.619746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application result.read() Jul 27 16:22:56.619746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 16:22:56.620681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 16:22:56.620681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 16:22:56.620681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 16:22:56.620681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 16:22:56.620681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 16:22:56.620681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 16:22:56.620681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 16:22:56.620681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 16:22:56.620681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:56.620681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 16:22:56.620681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:56.620681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:56.620681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 16:22:56.620681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 16:22:56.620681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 27 16:22:56.620681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return ref.to_dict() Jul 27 16:22:56.620681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 16:22:56.620681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application next(self.gen) Jul 27 16:22:56.622149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 16:22:56.622149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application yield resource Jul 27 16:22:56.622149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 16:22:56.622149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application next(self.gen) Jul 27 16:22:56.622149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 16:22:56.622149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 16:22:56.622149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 16:22:56.622149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 16:22:56.622149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 16:22:56.622149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise self.value Jul 27 16:22:56.622149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 16:22:56.622149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 16:22:56.622149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 16:22:56.622149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application session.commit() Jul 27 16:22:56.622149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 16:22:56.622149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 16:22:56.622149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 16:22:56.623417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 16:22:56.623417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 16:22:56.623417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self.session.flush() Jul 27 16:22:56.623417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 16:22:56.623417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 16:22:56.623417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 16:22:56.623417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 16:22:56.623417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 16:22:56.623417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application compat.raise_( Jul 27 16:22:56.623417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 16:22:56.623417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception Jul 27 16:22:56.623417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 16:22:56.623417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 16:22:56.623417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 16:22:56.623417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 16:22:56.623417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 16:22:56.623417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 16:22:56.624287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 16:22:56.624287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 16:22:56.624287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 27 16:22:56.624287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 27 16:22:56.624287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 16:22:56.624287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 16:22:56.624287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 16:22:56.624287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 16:22:56.624287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 16:22:56.624287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 16:22:56.624287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 16:22:56.624287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 16:22:56.624287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 16:22:56.624287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 16:22:56.624287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 16:22:56.624287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception Jul 27 16:22:56.625569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 16:22:56.625569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 16:22:56.625569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 16:22:56.625569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 16:22:56.625569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 16:22:56.625569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 16:22:56.625569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 16:22:56.625569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application conn.query(q) Jul 27 16:22:56.625569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 16:22:56.625569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 16:22:56.625569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 16:22:56.625569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application result.read() Jul 27 16:22:56.625569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 16:22:56.625569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 16:22:56.625569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 16:22:56.625569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 16:22:56.627005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 16:22:56.627005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 16:22:56.627005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 16:22:56.627005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 16:22:56.627005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 16:22:56.627005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 27 16:22:56.627005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [parameters: {'id': '6c8f92dc9b3d420ca0637cfc49ba0e05', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 27 16:22:56.627005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 16:22:56.627005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:56.627005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:22:56.627005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:56.627005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:56.627005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:56.627005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:56.627005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:56.627005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:56.628983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:56.628983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:56.628983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:56.628983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:56.628983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:56.628983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:56.628983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 27 16:22:56.628983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 27 16:22:56.628983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:56.628983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:56.628983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 27 16:22:56.628983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 27 16:22:56.628983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 16:22:56.628983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 16:22:56.628983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 27 16:22:56.628983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:56.631036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 240/477] 10.222.0.74 () {60 vars in 1131 bytes} [Wed Jul 27 16:22:56 2022] POST /identity/v3/roles => generated 144 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:22:56.631036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fbace-3c1c-4fc4-8004-eadbdf362c43 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:56.669926 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6b024-21f2-49e2-8399-9cf099c425ce None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:56.710524 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c6f67-ec02-4b1e-a7f5-096e40296881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:56.711612 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c6f67-ec02-4b1e-a7f5-096e40296881 None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:56.712085 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c6f67-ec02-4b1e-a7f5-096e40296881 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:56.712593 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c6f67-ec02-4b1e-a7f5-096e40296881 None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:56.713622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c6f67-ec02-4b1e-a7f5-096e40296881 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:56.715449 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c6f67-ec02-4b1e-a7f5-096e40296881 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:56.741939 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 238/478] 10.222.0.74 () {60 vars in 1132 bytes} [Wed Jul 27 16:22:56 2022] POST /identity/v3/users => generated 249 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:22:56.759557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fbace-3c1c-4fc4-8004-eadbdf362c43 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:56.799181 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ec803-23ad-4522-9d22-185d8520b5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:56.799760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ec803-23ad-4522-9d22-185d8520b5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:56.800042 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ec803-23ad-4522-9d22-185d8520b5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:56.800313 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ec803-23ad-4522-9d22-185d8520b5b2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:56.813259 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ec803-23ad-4522-9d22-185d8520b5b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:56.818362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-c63ec803-23ad-4522-9d22-185d8520b5b2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:56.818362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:56.818362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:56.818362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:56.818362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:56.818362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:56.818362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:56.818362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:56.818362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:56.818362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:56.818362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:56.818362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:56.818362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:56.818362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:56.818362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:56.818362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:56.819261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:56.819261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:56.819261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:56.819261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:56.819261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:56.819261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:56.819261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:56.819261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:56.819261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:56.819261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:56.819261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:56.819261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:56.819261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:56.819261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:56.819261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:56.819261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:56.820027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:56.820027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:56.820027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:56.820027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:56.820027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:56.820027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:56.820027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:56.820027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:56.820027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:56.820027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:56.820027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 16:22:56.820027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:56.820027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 241/479] 10.222.0.74 () {56 vars in 1114 bytes} [Wed Jul 27 16:22:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:22:56.826511 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c6f67-ec02-4b1e-a7f5-096e40296881 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:56.879580 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:56.966553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c6f67-ec02-4b1e-a7f5-096e40296881 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:57.003567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac99c5-59ab-4b4a-806b-27b896b3e919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:57.004183 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac99c5-59ab-4b4a-806b-27b896b3e919 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:57.004460 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac99c5-59ab-4b4a-806b-27b896b3e919 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:57.004716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac99c5-59ab-4b4a-806b-27b896b3e919 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:57.005521 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac99c5-59ab-4b4a-806b-27b896b3e919 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:57.005822 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac99c5-59ab-4b4a-806b-27b896b3e919 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:57.007260 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac99c5-59ab-4b4a-806b-27b896b3e919 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:57.015851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 239/480] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:22:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:57.836331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68132e84-3b36-4126-a595-e883935065aa None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:57.837281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68132e84-3b36-4126-a595-e883935065aa None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:57.837740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68132e84-3b36-4126-a595-e883935065aa None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:57.840255 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 242/481] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:57.846029 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb0cbc-1bbf-4162-9643-d3c5de4c8580 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:57.846444 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb0cbc-1bbf-4162-9643-d3c5de4c8580 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:57.846585 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb0cbc-1bbf-4162-9643-d3c5de4c8580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:57.873911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-bbdb0cbc-1bbf-4162-9643-d3c5de4c8580 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:57.875111 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-bbdb0cbc-1bbf-4162-9643-d3c5de4c8580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:57.967777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 240/482] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:58.002650 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68132e84-3b36-4126-a595-e883935065aa None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:58.036411 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:58.166678 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68132e84-3b36-4126-a595-e883935065aa None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:58.210471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca32bde-577d-41d6-b8b1-a47e0790cbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:58.211299 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca32bde-577d-41d6-b8b1-a47e0790cbc9 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:58.211586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca32bde-577d-41d6-b8b1-a47e0790cbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:58.211859 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca32bde-577d-41d6-b8b1-a47e0790cbc9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:58.225834 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca32bde-577d-41d6-b8b1-a47e0790cbc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:58.230786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-eca32bde-577d-41d6-b8b1-a47e0790cbc9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:22:58.230786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:58.230786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:58.230786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:58.230786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:58.230786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:58.230786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:58.230786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:58.230786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:58.230786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:58.230786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:58.230786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:58.230786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:58.230786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:58.230786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:58.230786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:58.231629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:58.231629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:58.231629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:58.231629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:58.231629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:58.231629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:58.231629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:58.231629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:58.231629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:58.231629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:58.231629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:58.231629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:58.231629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:58.231629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:58.231629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:58.231629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:58.232409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:58.232409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:58.232409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:58.232409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:58.232409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:58.232409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:58.232409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:58.232409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:58.232409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:58.232409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:58.232409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:22:58.232409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:58.232409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 243/483] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:22:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:58.239046 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb0cbc-1bbf-4162-9643-d3c5de4c8580 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:58.289960 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:58.395512 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb0cbc-1bbf-4162-9643-d3c5de4c8580 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:58.452138 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3ad09-4474-4f7f-b531-4e8521b6007e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:58.452769 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3ad09-4474-4f7f-b531-4e8521b6007e None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:58.453082 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3ad09-4474-4f7f-b531-4e8521b6007e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:58.453354 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3ad09-4474-4f7f-b531-4e8521b6007e None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:58.454084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3ad09-4474-4f7f-b531-4e8521b6007e None admin] RBAC: Adding query filter params (name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:58.454391 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3ad09-4474-4f7f-b531-4e8521b6007e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:58.455942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3ad09-4474-4f7f-b531-4e8521b6007e None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:58.469124 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 241/484] 10.222.0.74 () {56 vars in 1103 bytes} [Wed Jul 27 16:22:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:58.474611 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca32bde-577d-41d6-b8b1-a47e0790cbc9 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:58.589243 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:58.610432 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca32bde-577d-41d6-b8b1-a47e0790cbc9 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:58.637647 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250782c-27bc-482e-9bc7-166d787f225c None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:58.637937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250782c-27bc-482e-9bc7-166d787f225c None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:58.638201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250782c-27bc-482e-9bc7-166d787f225c None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:58.639548 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 242/485] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:22:58.644496 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66414c20-9538-4da9-9e95-4b896934b1c8 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:58.644779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66414c20-9538-4da9-9e95-4b896934b1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:58.645126 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66414c20-9538-4da9-9e95-4b896934b1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:58.673671 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-66414c20-9538-4da9-9e95-4b896934b1c8 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:58.674867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-66414c20-9538-4da9-9e95-4b896934b1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:58.680049 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead12a6-e43e-4f6b-87ef-8937b7c73423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:58.680701 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead12a6-e43e-4f6b-87ef-8937b7c73423 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:58.681009 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead12a6-e43e-4f6b-87ef-8937b7c73423 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:58.681289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead12a6-e43e-4f6b-87ef-8937b7c73423 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:58.702362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead12a6-e43e-4f6b-87ef-8937b7c73423 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:58.703749 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead12a6-e43e-4f6b-87ef-8937b7c73423 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:58.729685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-9ead12a6-e43e-4f6b-87ef-8937b7c73423 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:22:58.729685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:58.729685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:58.729685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:58.729685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:58.729685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:58.729685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:58.729685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:58.729685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:58.729685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:58.729685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:58.729685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:58.729685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:22:58.729685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:22:58.729685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:22:58.730621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:22:58.730621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:58.730621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:58.730621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:58.730621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:58.730621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:58.730621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:58.730621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:58.730621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:58.730621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:58.730621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:58.730621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:58.730621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:58.730621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:58.730621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:58.730621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:58.730621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:58.731430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:22:58.731430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:22:58.731430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:22:58.731430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:58.731430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 244/486] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:22:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:58.736139 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead12a6-e43e-4f6b-87ef-8937b7c73423 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:58.823353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 243/487] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:22:58.848463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66414c20-9538-4da9-9e95-4b896934b1c8 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:58.849343 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead12a6-e43e-4f6b-87ef-8937b7c73423 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:58.890835 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:58.907277 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:58.920270 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c54b94-6731-4bab-9a1b-b11fd2953033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:58.920920 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c54b94-6731-4bab-9a1b-b11fd2953033 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:58.921196 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c54b94-6731-4bab-9a1b-b11fd2953033 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:58.921513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c54b94-6731-4bab-9a1b-b11fd2953033 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:58.922218 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c54b94-6731-4bab-9a1b-b11fd2953033 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:58.922521 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c54b94-6731-4bab-9a1b-b11fd2953033 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:58.923902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c54b94-6731-4bab-9a1b-b11fd2953033 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:58.946176 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 245/488] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:58.952300 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c54b94-6731-4bab-9a1b-b11fd2953033 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:59.054395 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66414c20-9538-4da9-9e95-4b896934b1c8 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:59.056084 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c54b94-6731-4bab-9a1b-b11fd2953033 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:59.093352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71035d4-b36f-4fbc-9665-35ddd0e3d57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:59.094357 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71035d4-b36f-4fbc-9665-35ddd0e3d57f None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:59.094975 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71035d4-b36f-4fbc-9665-35ddd0e3d57f None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:59.095483 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71035d4-b36f-4fbc-9665-35ddd0e3d57f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:59.104351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71035d4-b36f-4fbc-9665-35ddd0e3d57f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:59.105011 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacbd71-a78a-44c2-93ab-c5b2ca119511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:59.106190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacbd71-a78a-44c2-93ab-c5b2ca119511 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:59.106472 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacbd71-a78a-44c2-93ab-c5b2ca119511 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:59.106745 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacbd71-a78a-44c2-93ab-c5b2ca119511 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:59.107695 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71035d4-b36f-4fbc-9665-35ddd0e3d57f None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:59.117450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacbd71-a78a-44c2-93ab-c5b2ca119511 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:59.122962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-2cacbd71-a78a-44c2-93ab-c5b2ca119511 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:22:59.122962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:59.122962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:59.122962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:59.122962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:59.122962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:59.122962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:59.122962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:59.122962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:59.122962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:59.122962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:59.122962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:59.122962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:59.122962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:59.122962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:59.122962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:59.123828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:59.123828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:59.123828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:59.123828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:59.123828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:59.123828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:59.123828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:59.123828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:59.123828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:59.123828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:59.123828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:59.123828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:59.123828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:59.123828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:59.123828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:59.123828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:59.124588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:59.124588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:59.124588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:59.124588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:59.124588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:59.124588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:59.124588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:59.124588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:59.124588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:59.124588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:59.124588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:22:59.124588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:59.124588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 244/489] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:22:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:59.126597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-b71035d4-b36f-4fbc-9665-35ddd0e3d57f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 16:22:59.126597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:59.126597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:59.126597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:59.126597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:59.126597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:59.126597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:59.126597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:59.126597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:59.126597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:59.126597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:59.126597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:59.126597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:22:59.126597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:22:59.126597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:22:59.126597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:22:59.127413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:59.127413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:59.127413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:22:59.127413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:22:59.127413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:22:59.127413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:22:59.127413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:59.127413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:59.127413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:59.127413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:59.127413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:59.127413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:59.127413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:59.127413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:59.127413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:59.127413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:59.127413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:59.127413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:59.128282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:59.128282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:59.128282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:22:59.128282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:22:59.128282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:22:59.128282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:22:59.128282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:22:59.128282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:22:59.128282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 16:22:59.128282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:22:59.129783 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5334d-56ce-4233-bca2-77a5dca8665c None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:59.130079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5334d-56ce-4233-bca2-77a5dca8665c None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:59.130362 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5334d-56ce-4233-bca2-77a5dca8665c None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:59.131349 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 246/490] 10.222.0.74 () {56 vars in 1093 bytes} [Wed Jul 27 16:22:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:59.132628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71035d4-b36f-4fbc-9665-35ddd0e3d57f None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:59.137883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 245/491] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:59 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:59.138389 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5334d-56ce-4233-bca2-77a5dca8665c None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:59.194965 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:59.229843 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:59.252189 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71035d4-b36f-4fbc-9665-35ddd0e3d57f None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:59.258753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5334d-56ce-4233-bca2-77a5dca8665c None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:59.287981 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5a7e3-5f38-4e06-8a9a-cd0dee86f3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:59.288612 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5a7e3-5f38-4e06-8a9a-cd0dee86f3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:59.288904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5a7e3-5f38-4e06-8a9a-cd0dee86f3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:59.289203 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5a7e3-5f38-4e06-8a9a-cd0dee86f3e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:59.289917 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5a7e3-5f38-4e06-8a9a-cd0dee86f3e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:59.290236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5a7e3-5f38-4e06-8a9a-cd0dee86f3e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:59.291697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5a7e3-5f38-4e06-8a9a-cd0dee86f3e3 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:59.300960 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 247/492] 10.222.0.74 () {56 vars in 1103 bytes} [Wed Jul 27 16:22:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:22:59.302461 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636aaba-13f5-4baa-8e14-c81decb10f43 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:59.302939 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636aaba-13f5-4baa-8e14-c81decb10f43 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:59.303459 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636aaba-13f5-4baa-8e14-c81decb10f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:59.310846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a7802-1628-4573-9fea-2670dc83840f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:59.311418 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a7802-1628-4573-9fea-2670dc83840f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:59.311699 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a7802-1628-4573-9fea-2670dc83840f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:59.311976 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a7802-1628-4573-9fea-2670dc83840f None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:59.312781 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a7802-1628-4573-9fea-2670dc83840f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:59.313121 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a7802-1628-4573-9fea-2670dc83840f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:59.315920 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a7802-1628-4573-9fea-2670dc83840f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:59.326887 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-3636aaba-13f5-4baa-8e14-c81decb10f43 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:59.329348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-3636aaba-13f5-4baa-8e14-c81decb10f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:59.364969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 246/493] 10.222.0.74 () {56 vars in 1133 bytes} [Wed Jul 27 16:22:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:59.366166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a7802-1628-4573-9fea-2670dc83840f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:59.414233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 248/494] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:59.415434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636aaba-13f5-4baa-8e14-c81decb10f43 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:59.471414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:59.478145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a7802-1628-4573-9fea-2670dc83840f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:59.512292 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab6355-e562-48e8-bd7b-ec9b2618a488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:59.512950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab6355-e562-48e8-bd7b-ec9b2618a488 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:59.513235 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab6355-e562-48e8-bd7b-ec9b2618a488 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:59.513507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab6355-e562-48e8-bd7b-ec9b2618a488 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:59.520120 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:59.525491 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab6355-e562-48e8-bd7b-ec9b2618a488 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:59.526966 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab6355-e562-48e8-bd7b-ec9b2618a488 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:59.528868 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636aaba-13f5-4baa-8e14-c81decb10f43 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:59.542622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-deab6355-e562-48e8-bd7b-ec9b2618a488 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:22:59.542622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:59.542622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:59.542622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:59.542622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:59.542622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:59.542622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:59.542622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:59.542622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:59.542622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:59.542622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:59.542622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:59.542622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:22:59.542622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:22:59.542622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:22:59.543581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:22:59.543581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:59.543581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:59.543581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:59.543581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:59.543581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:59.543581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:59.543581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:59.543581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:59.543581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:59.543581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:59.543581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:59.543581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:59.543581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:59.543581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:59.543581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:59.543581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:59.544407 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:22:59.544407 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:22:59.544407 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:22:59.544407 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:59.544407 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 247/495] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:22:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:59.545217 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf04bcc-9f1e-489d-9489-d0dfe553d811 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:59.545315 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf04bcc-9f1e-489d-9489-d0dfe553d811 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:59.545585 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf04bcc-9f1e-489d-9489-d0dfe553d811 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:59.549010 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 248/496] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:59.549010 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf04bcc-9f1e-489d-9489-d0dfe553d811 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:59.572072 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a23648-1a64-43c4-9155-11b7cd8acefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:59.572623 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a23648-1a64-43c4-9155-11b7cd8acefc None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:59.572927 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a23648-1a64-43c4-9155-11b7cd8acefc None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:59.573203 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a23648-1a64-43c4-9155-11b7cd8acefc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:59.646288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a23648-1a64-43c4-9155-11b7cd8acefc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:59.647734 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a23648-1a64-43c4-9155-11b7cd8acefc None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:59.662875 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf04bcc-9f1e-489d-9489-d0dfe553d811 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:59.697282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:59.700778 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696617a2-7468-4324-9c4a-b4a81a18961e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:59.701607 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696617a2-7468-4324-9c4a-b4a81a18961e None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:59.701894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696617a2-7468-4324-9c4a-b4a81a18961e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:59.702169 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696617a2-7468-4324-9c4a-b4a81a18961e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:59.712927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696617a2-7468-4324-9c4a-b4a81a18961e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:22:59.718957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-696617a2-7468-4324-9c4a-b4a81a18961e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:22:59.718957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:22:59.718957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:22:59.718957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:22:59.718957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:22:59.718957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:22:59.718957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:22:59.718957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:22:59.718957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:22:59.718957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:22:59.718957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:22:59.718957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:22:59.718957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:22:59.718957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:22:59.718957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:22:59.718957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:22:59.719851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:22:59.719851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:22:59.719851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:22:59.719851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:22:59.719851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:22:59.719851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:22:59.719851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:22:59.719851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:22:59.719851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:22:59.719851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:22:59.719851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:22:59.719851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:22:59.719851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:22:59.719851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:22:59.719851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:22:59.719851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:22:59.720706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:22:59.720706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:22:59.720706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:22:59.720706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:22:59.720706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:22:59.720706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:22:59.720706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:22:59.720706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:22:59.720706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:22:59.720706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:22:59.720706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:22:59.720706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:22:59.720706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 249/497] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:22:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:22:59.721997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696617a2-7468-4324-9c4a-b4a81a18961e None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:59.769250 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 249/498] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:22:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:22:59.769250 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7dcbb3-690c-4709-967b-333c487c9617 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:59.769250 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7dcbb3-690c-4709-967b-333c487c9617 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:59.769250 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7dcbb3-690c-4709-967b-333c487c9617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:59.771538 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:59.786623 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-5c7dcbb3-690c-4709-967b-333c487c9617 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:22:59.787802 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-5c7dcbb3-690c-4709-967b-333c487c9617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:22:59.823873 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696617a2-7468-4324-9c4a-b4a81a18961e None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:59.854021 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 250/499] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:22:59.854834 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7dcbb3-690c-4709-967b-333c487c9617 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:59.857562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0585214-5602-4105-b912-07c1cde1d253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:59.858179 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0585214-5602-4105-b912-07c1cde1d253 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:59.858463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0585214-5602-4105-b912-07c1cde1d253 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:59.858743 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0585214-5602-4105-b912-07c1cde1d253 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:59.859431 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0585214-5602-4105-b912-07c1cde1d253 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:59.859738 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0585214-5602-4105-b912-07c1cde1d253 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:59.861152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0585214-5602-4105-b912-07c1cde1d253 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:22:59.881268 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 250/500] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:22:59.882753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4fcca-1f5d-4da4-815a-57d1fe74bd3d None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:59.883042 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4fcca-1f5d-4da4-815a-57d1fe74bd3d None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:59.883317 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4fcca-1f5d-4da4-815a-57d1fe74bd3d None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:59.884513 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 251/501] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:22:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:22:59.885231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4fcca-1f5d-4da4-815a-57d1fe74bd3d None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:22:59.933903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:22:59.961897 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7dcbb3-690c-4709-967b-333c487c9617 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:59.988747 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4fcca-1f5d-4da4-815a-57d1fe74bd3d None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:22:59.994921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6e084-f861-491b-8fd7-5e0356e99974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:22:59.995543 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6e084-f861-491b-8fd7-5e0356e99974 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:22:59.996319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6e084-f861-491b-8fd7-5e0356e99974 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:22:59.996591 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6e084-f861-491b-8fd7-5e0356e99974 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:22:59.997337 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6e084-f861-491b-8fd7-5e0356e99974 None admin] RBAC: Adding query filter params (name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:22:59.997643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6e084-f861-491b-8fd7-5e0356e99974 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:22:59.999030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6e084-f861-491b-8fd7-5e0356e99974 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:00.001448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:00.007569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 251/502] 10.222.0.74 () {56 vars in 1103 bytes} [Wed Jul 27 16:22:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:00.008358 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6e084-f861-491b-8fd7-5e0356e99974 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:00.025979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6eb082-cea2-4e6e-9ebf-df5dd7ccf23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:00.026621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6eb082-cea2-4e6e-9ebf-df5dd7ccf23d None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:00.026902 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6eb082-cea2-4e6e-9ebf-df5dd7ccf23d None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:00.027184 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6eb082-cea2-4e6e-9ebf-df5dd7ccf23d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:00.037076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6eb082-cea2-4e6e-9ebf-df5dd7ccf23d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:00.038533 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6eb082-cea2-4e6e-9ebf-df5dd7ccf23d None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:00.054894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-cb6eb082-cea2-4e6e-9ebf-df5dd7ccf23d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:00.054894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:00.054894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:00.054894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:00.054894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:00.054894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:00.054894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:00.054894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:00.054894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:00.054894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:00.054894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:00.054894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:00.054894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:00.054894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:00.054894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:00.055814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:00.055814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:00.055814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:00.055814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:00.055814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:00.055814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:00.055814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:00.055814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:00.055814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:00.055814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:00.055814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:00.055814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:00.055814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:00.055814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:00.055814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:00.055814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:00.055814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:00.056646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:00.056646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:00.056646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:00.056646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:00.056646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 252/503] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:22:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:00.056947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6eb082-cea2-4e6e-9ebf-df5dd7ccf23d None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:00.110766 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6e084-f861-491b-8fd7-5e0356e99974 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:00.143295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3fe4a1-2da6-4fc9-b30f-95c780430df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:00.143944 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fe4a1-2da6-4fc9-b30f-95c780430df0 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:00.144207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fe4a1-2da6-4fc9-b30f-95c780430df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:00.144475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fe4a1-2da6-4fc9-b30f-95c780430df0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:00.147042 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:00.155129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3fe4a1-2da6-4fc9-b30f-95c780430df0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:00.157618 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6eb082-cea2-4e6e-9ebf-df5dd7ccf23d None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:00.160473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-1a3fe4a1-2da6-4fc9-b30f-95c780430df0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:00.160473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:00.160473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:00.160473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:00.160473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:00.160473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:00.160473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:00.160473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:00.160473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:00.160473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:00.160473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:00.160473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:00.160473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:00.160473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:00.160473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:00.160473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:00.161414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:00.161414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:00.161414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:00.161414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:00.161414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:00.161414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:00.161414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:00.161414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:00.161414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:00.161414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:00.161414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:00.161414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:00.161414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:00.161414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:00.161414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:00.161414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:00.162204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:00.162204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:00.162204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:00.162204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:00.162204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:00.162204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:00.162204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:00.162204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:00.162204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:00.162204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:00.162204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:00.162204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:00.162204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 252/504] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:23:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:00.163330 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4e5e8-9541-4cf4-a720-8d091bbde08e None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:00.163549 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4e5e8-9541-4cf4-a720-8d091bbde08e None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:00.163824 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4e5e8-9541-4cf4-a720-8d091bbde08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:00.181738 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-f4f4e5e8-9541-4cf4-a720-8d091bbde08e None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:00.182925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f4f4e5e8-9541-4cf4-a720-8d091bbde08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:00.194617 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80368bde-9a7b-4a8e-8ad2-778202b51e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:00.195251 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80368bde-9a7b-4a8e-8ad2-778202b51e74 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:00.195525 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80368bde-9a7b-4a8e-8ad2-778202b51e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:00.195793 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80368bde-9a7b-4a8e-8ad2-778202b51e74 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:00.203494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80368bde-9a7b-4a8e-8ad2-778202b51e74 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:00.204948 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80368bde-9a7b-4a8e-8ad2-778202b51e74 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:00.217884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-80368bde-9a7b-4a8e-8ad2-778202b51e74 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 16:23:00.217884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:00.217884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:00.217884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:00.217884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:00.217884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:00.217884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:00.217884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:00.217884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:00.217884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:00.217884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:00.217884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:00.217884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:00.217884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:00.217884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:00.217884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:00.218969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:00.218969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:00.218969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:00.218969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:00.218969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:00.218969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:00.218969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:00.218969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:00.218969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:00.218969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:00.218969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:00.218969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:00.218969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:00.218969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:00.218969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:00.218969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:00.218969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:00.218969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:00.220927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:00.220927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:00.220927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:00.220927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:00.220927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:00.220927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:00.220927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:00.220927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:00.220927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 16:23:00.220927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:00.220927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 253/505] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:23:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:00.220927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80368bde-9a7b-4a8e-8ad2-778202b51e74 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:00.247353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:00.253250 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 253/506] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:23:00.254101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4e5e8-9541-4cf4-a720-8d091bbde08e None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:00.326821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80368bde-9a7b-4a8e-8ad2-778202b51e74 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:00.354299 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4e5e8-9541-4cf4-a720-8d091bbde08e None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:00.359488 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89611271-1459-493b-be47-62b1bec7d24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:00.360129 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89611271-1459-493b-be47-62b1bec7d24f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:00.360413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89611271-1459-493b-be47-62b1bec7d24f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:00.360688 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89611271-1459-493b-be47-62b1bec7d24f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:00.371578 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89611271-1459-493b-be47-62b1bec7d24f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:00.372993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89611271-1459-493b-be47-62b1bec7d24f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:00.380531 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:00.387207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-89611271-1459-493b-be47-62b1bec7d24f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:00.387207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:00.387207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:00.387207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:00.387207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:00.387207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:00.387207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:00.387207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:00.387207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:00.387207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:00.387207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:00.387207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:00.387207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:00.387207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:00.387207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:00.388061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:00.388061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:00.388061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:00.388061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:00.388061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:00.388061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:00.388061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:00.388061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:00.388061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:00.388061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:00.388061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:00.388061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:00.388061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:00.388061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:00.388061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:00.388061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:00.388061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:00.388955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:00.388955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:00.388955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:00.388955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:00.388955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 254/507] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:00.389623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89611271-1459-493b-be47-62b1bec7d24f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:00.394675 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962cbc3a-6078-464e-a610-ad9406ab3242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:00.395308 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cbc3a-6078-464e-a610-ad9406ab3242 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:00.395577 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cbc3a-6078-464e-a610-ad9406ab3242 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:00.396057 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cbc3a-6078-464e-a610-ad9406ab3242 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:00.397305 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962cbc3a-6078-464e-a610-ad9406ab3242 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:00.397863 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962cbc3a-6078-464e-a610-ad9406ab3242 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:00.400123 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962cbc3a-6078-464e-a610-ad9406ab3242 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:00.412768 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 254/508] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:00.413635 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962cbc3a-6078-464e-a610-ad9406ab3242 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:00.469682 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:00.495671 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89611271-1459-493b-be47-62b1bec7d24f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:00.518585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962cbc3a-6078-464e-a610-ad9406ab3242 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:00.528293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb872b3-363a-4f75-921d-e5800b0b30f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:00.528912 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb872b3-363a-4f75-921d-e5800b0b30f7 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:00.529205 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb872b3-363a-4f75-921d-e5800b0b30f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:00.529479 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb872b3-363a-4f75-921d-e5800b0b30f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:00.530182 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb872b3-363a-4f75-921d-e5800b0b30f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:00.530483 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb872b3-363a-4f75-921d-e5800b0b30f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:00.531858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb872b3-363a-4f75-921d-e5800b0b30f7 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:00.539463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 255/509] 10.222.0.74 () {56 vars in 1103 bytes} [Wed Jul 27 16:23:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:00.540227 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb872b3-363a-4f75-921d-e5800b0b30f7 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:00.556616 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2545d50-c1b4-43a6-88c2-76993bb4b61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:00.557264 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2545d50-c1b4-43a6-88c2-76993bb4b61f None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:00.557547 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2545d50-c1b4-43a6-88c2-76993bb4b61f None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:00.557824 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2545d50-c1b4-43a6-88c2-76993bb4b61f None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:00.559275 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2545d50-c1b4-43a6-88c2-76993bb4b61f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:00.559575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2545d50-c1b4-43a6-88c2-76993bb4b61f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:00.561008 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2545d50-c1b4-43a6-88c2-76993bb4b61f None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:00.597673 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 255/510] 10.222.0.74 () {56 vars in 1137 bytes} [Wed Jul 27 16:23:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:00.598811 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2545d50-c1b4-43a6-88c2-76993bb4b61f None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:00.641692 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb872b3-363a-4f75-921d-e5800b0b30f7 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:00.643811 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:00.674488 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa5fa3-e37c-4f4d-bb85-f8164443e301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:00.675122 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa5fa3-e37c-4f4d-bb85-f8164443e301 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:00.675401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa5fa3-e37c-4f4d-bb85-f8164443e301 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:00.675672 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa5fa3-e37c-4f4d-bb85-f8164443e301 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:00.678300 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:00.687150 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa5fa3-e37c-4f4d-bb85-f8164443e301 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:00.688614 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa5fa3-e37c-4f4d-bb85-f8164443e301 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:00.696918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2545d50-c1b4-43a6-88c2-76993bb4b61f None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:00.703147 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-d5aa5fa3-e37c-4f4d-bb85-f8164443e301 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:00.703147 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:00.703147 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:00.703147 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:00.703147 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:00.703147 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:00.703147 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:00.703147 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:00.703147 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:00.703147 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:00.703147 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:00.703147 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:00.703147 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:00.703147 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:00.703147 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:00.704115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:00.704115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:00.704115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:00.704115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:00.704115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:00.704115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:00.704115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:00.704115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:00.704115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:00.704115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:00.704115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:00.704115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:00.704115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:00.704115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:00.704115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:00.704115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:00.704115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:00.705093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:00.705093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:00.705093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:00.705093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:00.705093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 256/511] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:00.705427 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa5fa3-e37c-4f4d-bb85-f8164443e301 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:00.738838 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffeafb-4e22-445f-8281-28c96c611707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:00.739461 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffeafb-4e22-445f-8281-28c96c611707 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:00.739740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffeafb-4e22-445f-8281-28c96c611707 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:00.740016 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffeafb-4e22-445f-8281-28c96c611707 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:00.740706 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffeafb-4e22-445f-8281-28c96c611707 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:00.741040 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffeafb-4e22-445f-8281-28c96c611707 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:00.742427 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffeafb-4e22-445f-8281-28c96c611707 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:00.754666 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 256/512] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:00.755535 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffeafb-4e22-445f-8281-28c96c611707 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:00.810956 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa5fa3-e37c-4f4d-bb85-f8164443e301 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:00.846727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a62eb-e06c-4a1f-8a03-91cd2e105903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:00.847365 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a62eb-e06c-4a1f-8a03-91cd2e105903 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:00.847648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a62eb-e06c-4a1f-8a03-91cd2e105903 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:00.847924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a62eb-e06c-4a1f-8a03-91cd2e105903 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:00.854650 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a62eb-e06c-4a1f-8a03-91cd2e105903 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:00.856172 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a62eb-e06c-4a1f-8a03-91cd2e105903 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:00.867862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-c70a62eb-e06c-4a1f-8a03-91cd2e105903 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:00.867862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:00.867862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:00.867862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:00.867862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:00.867862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:00.867862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:00.867862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:00.867862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:00.867862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:00.867862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:00.867862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:00.867862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:00.867862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:00.867862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:00.869211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:00.869211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:00.869211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:00.869211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:00.869211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:00.869211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:00.869211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:00.869211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:00.869211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:00.869211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:00.869211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:00.869211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:00.869211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:00.869211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:00.869211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:00.869211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:00.869211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:00.870063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:00.870063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:00.870063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:00.870063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:00.870063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:00.870063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:00.870063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:00.870063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:00.873311 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 257/513] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:00.873980 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffeafb-4e22-445f-8281-28c96c611707 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:00.874386 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a62eb-e06c-4a1f-8a03-91cd2e105903 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:00.876884 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:00.908593 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bea03-87f5-432e-a88d-37ba4275f9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:00.909198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bea03-87f5-432e-a88d-37ba4275f9f5 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:00.909474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bea03-87f5-432e-a88d-37ba4275f9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:00.909744 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bea03-87f5-432e-a88d-37ba4275f9f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:00.919955 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95bea03-87f5-432e-a88d-37ba4275f9f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:00.921422 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95bea03-87f5-432e-a88d-37ba4275f9f5 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:00.922085 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:00.936484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-a95bea03-87f5-432e-a88d-37ba4275f9f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:00.936484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:00.936484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:00.936484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:00.936484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:00.936484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:00.936484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:00.936484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:00.936484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:00.936484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:00.936484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:00.936484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:00.936484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:00.936484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:00.936484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:00.937351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:00.937351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:00.937351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:00.937351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:00.937351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:00.937351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:00.937351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:00.937351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:00.937351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:00.937351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:00.937351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:00.937351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:00.937351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:00.937351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:00.937351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:00.937351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:00.937351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:00.938479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:00.938479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:00.938479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:00.938479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:00.938479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 257/514] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:00.938479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bea03-87f5-432e-a88d-37ba4275f9f5 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:00.975297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a62eb-e06c-4a1f-8a03-91cd2e105903 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:01.008951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adaf71a-6785-484e-989a-9b342a33c725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:01.009567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adaf71a-6785-484e-989a-9b342a33c725 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:01.009844 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adaf71a-6785-484e-989a-9b342a33c725 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:01.010114 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adaf71a-6785-484e-989a-9b342a33c725 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:01.038404 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bea03-87f5-432e-a88d-37ba4275f9f5 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:01.072555 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a63bc3-c93f-44c1-955a-0ad30970eda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:01.073212 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a63bc3-c93f-44c1-955a-0ad30970eda5 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:01.073496 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a63bc3-c93f-44c1-955a-0ad30970eda5 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:01.073776 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a63bc3-c93f-44c1-955a-0ad30970eda5 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:01.075051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a63bc3-c93f-44c1-955a-0ad30970eda5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:01.075621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a63bc3-c93f-44c1-955a-0ad30970eda5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:01.075711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adaf71a-6785-484e-989a-9b342a33c725 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:01.077027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a63bc3-c93f-44c1-955a-0ad30970eda5 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:01.077163 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adaf71a-6785-484e-989a-9b342a33c725 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:01.089298 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 258/515] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:01.090135 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a63bc3-c93f-44c1-955a-0ad30970eda5 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:01.107665 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:01.135087 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:01.194162 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 258/516] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:23:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:23:01.194162 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adaf71a-6785-484e-989a-9b342a33c725 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:01.199248 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a63bc3-c93f-44c1-955a-0ad30970eda5 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:01.232831 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d84be-52d9-41e1-9202-130627896f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:01.233511 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d84be-52d9-41e1-9202-130627896f92 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:01.233786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d84be-52d9-41e1-9202-130627896f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:01.234055 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d84be-52d9-41e1-9202-130627896f92 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:01.240837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d84be-52d9-41e1-9202-130627896f92 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:01.242313 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d84be-52d9-41e1-9202-130627896f92 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:01.254460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-6c7d84be-52d9-41e1-9202-130627896f92 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 16:23:01.254460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:01.254460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:01.254460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:01.254460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:01.254460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:01.254460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:01.254460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:01.254460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:01.254460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:01.254460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:01.254460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:01.254460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:01.254460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:01.254460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:01.254460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:01.255409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:01.255409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:01.255409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:01.255409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:01.255409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:01.255409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:01.255409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:01.255409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:01.255409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:01.255409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:01.255409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:01.255409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:01.255409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:01.255409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:01.255409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:01.255409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:01.255409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:01.255409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:01.256285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:01.256285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:01.256285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:01.256285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:01.256285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:01.256285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:01.256285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:01.256285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:01.256285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 16:23:01.256285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:01.256285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 259/517] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:23:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:01.256830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d84be-52d9-41e1-9202-130627896f92 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:01.290792 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adaf71a-6785-484e-989a-9b342a33c725 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:01.324879 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ed770-6910-43f0-9289-ab34e77ef5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:01.325804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ed770-6910-43f0-9289-ab34e77ef5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:01.326088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ed770-6910-43f0-9289-ab34e77ef5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:01.326363 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ed770-6910-43f0-9289-ab34e77ef5e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:01.327112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ed770-6910-43f0-9289-ab34e77ef5e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:01.327423 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ed770-6910-43f0-9289-ab34e77ef5e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:01.328883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ed770-6910-43f0-9289-ab34e77ef5e6 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:01.332360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:01.346283 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 259/518] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:01.347075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ed770-6910-43f0-9289-ab34e77ef5e6 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:01.359445 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:01.360310 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d84be-52d9-41e1-9202-130627896f92 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:01.393606 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c37bb7-d8f6-4430-918c-a726dd03a954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:01.394246 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c37bb7-d8f6-4430-918c-a726dd03a954 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:01.394524 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c37bb7-d8f6-4430-918c-a726dd03a954 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:01.394796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c37bb7-d8f6-4430-918c-a726dd03a954 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:01.395478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c37bb7-d8f6-4430-918c-a726dd03a954 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:01.395779 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c37bb7-d8f6-4430-918c-a726dd03a954 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:01.397178 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c37bb7-d8f6-4430-918c-a726dd03a954 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:01.409197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 260/519] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:01.409973 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c37bb7-d8f6-4430-918c-a726dd03a954 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:01.451706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ed770-6910-43f0-9289-ab34e77ef5e6 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:01.485307 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfdf01f-cefd-4b0b-88d7-c3e50f1bf8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:01.485962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdf01f-cefd-4b0b-88d7-c3e50f1bf8c0 None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:01.486250 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdf01f-cefd-4b0b-88d7-c3e50f1bf8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:01.486527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdf01f-cefd-4b0b-88d7-c3e50f1bf8c0 None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:01.487207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfdf01f-cefd-4b0b-88d7-c3e50f1bf8c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:01.488659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfdf01f-cefd-4b0b-88d7-c3e50f1bf8c0 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:01.524920 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 260/520] 10.222.0.74 () {60 vars in 1132 bytes} [Wed Jul 27 16:23:01 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:23:01.524920 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfdf01f-cefd-4b0b-88d7-c3e50f1bf8c0 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:01.525208 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c37bb7-d8f6-4430-918c-a726dd03a954 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:01.556866 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b0d00-953d-4845-81f1-95b97b8207c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:01.557610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b0d00-953d-4845-81f1-95b97b8207c9 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:01.557929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b0d00-953d-4845-81f1-95b97b8207c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:01.558244 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b0d00-953d-4845-81f1-95b97b8207c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:01.563061 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:01.574010 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b0d00-953d-4845-81f1-95b97b8207c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:01.575734 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b0d00-953d-4845-81f1-95b97b8207c9 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:01.582390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:01.593434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-9f8b0d00-953d-4845-81f1-95b97b8207c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:01.593434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:01.593434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:01.593434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:01.593434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:01.593434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:01.593434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:01.593434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:01.593434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:01.593434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:01.593434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:01.593434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:01.593434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:01.593434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:01.593434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:01.594372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:01.594372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:01.594372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:01.594372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:01.594372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:01.594372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:01.594372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:01.594372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:01.594372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:01.594372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:01.594372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:01.594372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:01.594372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:01.594372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:01.594372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:01.594372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:01.594372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:01.595246 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:01.595246 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:01.595246 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:01.595246 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:01.595246 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 261/521] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:01.595524 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b0d00-953d-4845-81f1-95b97b8207c9 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:01.634119 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfdf01f-cefd-4b0b-88d7-c3e50f1bf8c0 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:01.674048 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3a01b-8ea7-4174-8c75-b6b67c1ee05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:01.675052 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3a01b-8ea7-4174-8c75-b6b67c1ee05b None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:01.675334 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3a01b-8ea7-4174-8c75-b6b67c1ee05b None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:01.675608 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3a01b-8ea7-4174-8c75-b6b67c1ee05b None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:01.676402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3a01b-8ea7-4174-8c75-b6b67c1ee05b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:01.676711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3a01b-8ea7-4174-8c75-b6b67c1ee05b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:01.678202 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3a01b-8ea7-4174-8c75-b6b67c1ee05b None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:01.703297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 261/522] 10.222.0.74 () {56 vars in 1137 bytes} [Wed Jul 27 16:23:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:01.703937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3a01b-8ea7-4174-8c75-b6b67c1ee05b None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:01.713718 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b0d00-953d-4845-81f1-95b97b8207c9 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:01.749847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ad3e2-807d-4d44-aca9-d494d685fbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:01.750561 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ad3e2-807d-4d44-aca9-d494d685fbfa None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:01.750844 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ad3e2-807d-4d44-aca9-d494d685fbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:01.751116 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ad3e2-807d-4d44-aca9-d494d685fbfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:01.751899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ad3e2-807d-4d44-aca9-d494d685fbfa None admin] RBAC: Adding query filter params (role.id=2aa0cdfed849496db8f3692c9569d7ee, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=56dfafd15e284b7986ba07dd3819bf65) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:01.752206 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ad3e2-807d-4d44-aca9-d494d685fbfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:01.753652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ad3e2-807d-4d44-aca9-d494d685fbfa None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:01.759556 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 262/523] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:23:01 2022] GET /identity/v3/role_assignments?user.id=56dfafd15e284b7986ba07dd3819bf65&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=2aa0cdfed849496db8f3692c9569d7ee => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:01.760344 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ad3e2-807d-4d44-aca9-d494d685fbfa None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:01.821255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3a01b-8ea7-4174-8c75-b6b67c1ee05b None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:01.825663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:01.849579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:01.862466 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45deddf-6536-4ed9-b283-da2720dd8415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:01.863138 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45deddf-6536-4ed9-b283-da2720dd8415 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:01.863438 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45deddf-6536-4ed9-b283-da2720dd8415 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:01.863704 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45deddf-6536-4ed9-b283-da2720dd8415 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:01.872696 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45deddf-6536-4ed9-b283-da2720dd8415 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:01.874481 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45deddf-6536-4ed9-b283-da2720dd8415 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:01.885512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ad3e2-807d-4d44-aca9-d494d685fbfa None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:01.888644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-d45deddf-6536-4ed9-b283-da2720dd8415 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 16:23:01.888644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:01.888644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:01.888644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:01.888644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:01.888644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:01.888644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:01.888644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:01.888644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:01.888644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:01.888644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:01.888644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:01.888644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:01.888644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:01.888644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:01.888644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:01.889760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:01.889760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:01.889760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:01.889760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:01.889760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:01.889760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:01.889760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:01.889760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:01.889760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:01.889760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:01.889760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:01.889760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:01.889760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:01.889760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:01.889760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:01.889760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:01.889760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:01.889760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:01.890945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:01.890945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:01.890945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:01.890945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:01.890945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:01.890945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:01.890945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:01.890945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:01.890945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 16:23:01.890945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:01.890945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 262/524] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:23:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:01.890945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45deddf-6536-4ed9-b283-da2720dd8415 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:01.928718 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801ffb8-9439-4425-ace1-29a5fdeb4ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:01.929634 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801ffb8-9439-4425-ace1-29a5fdeb4ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:01.930085 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801ffb8-9439-4425-ace1-29a5fdeb4ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:01.930499 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801ffb8-9439-4425-ace1-29a5fdeb4ae5 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:01.931435 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801ffb8-9439-4425-ace1-29a5fdeb4ae5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:01.931930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801ffb8-9439-4425-ace1-29a5fdeb4ae5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:01.933623 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801ffb8-9439-4425-ace1-29a5fdeb4ae5 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:01.947192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 263/525] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:01.948352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801ffb8-9439-4425-ace1-29a5fdeb4ae5 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:02.013737 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45deddf-6536-4ed9-b283-da2720dd8415 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:02.056503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3e529-6b42-4eec-89dd-3d0d58baf40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:02.057444 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3e529-6b42-4eec-89dd-3d0d58baf40b None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:02.058457 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3e529-6b42-4eec-89dd-3d0d58baf40b None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:02.058888 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3e529-6b42-4eec-89dd-3d0d58baf40b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:02.072662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801ffb8-9439-4425-ace1-29a5fdeb4ae5 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:02.089746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:02.112491 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b197f-5ca3-429a-bcf7-1fe27888a84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:02.113204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b197f-5ca3-429a-bcf7-1fe27888a84b None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:02.113489 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b197f-5ca3-429a-bcf7-1fe27888a84b None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:02.113767 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b197f-5ca3-429a-bcf7-1fe27888a84b None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:02.114566 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b197f-5ca3-429a-bcf7-1fe27888a84b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:02.114926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b197f-5ca3-429a-bcf7-1fe27888a84b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:02.116383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b197f-5ca3-429a-bcf7-1fe27888a84b None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:02.128644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:02.135891 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 264/526] 10.222.0.74 () {56 vars in 1139 bytes} [Wed Jul 27 16:23:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:02.137258 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b197f-5ca3-429a-bcf7-1fe27888a84b None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:02.155992 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3e529-6b42-4eec-89dd-3d0d58baf40b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:02.158357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3e529-6b42-4eec-89dd-3d0d58baf40b None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:02.262696 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b197f-5ca3-429a-bcf7-1fe27888a84b None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:02.275214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 263/527] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:23:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:23:02.276340 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3e529-6b42-4eec-89dd-3d0d58baf40b None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:02.298715 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d541dc1-8e71-4427-912b-754e0820a2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:02.299409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d541dc1-8e71-4427-912b-754e0820a2fd None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:02.299698 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d541dc1-8e71-4427-912b-754e0820a2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:02.299976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d541dc1-8e71-4427-912b-754e0820a2fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:02.307045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d541dc1-8e71-4427-912b-754e0820a2fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:02.309115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d541dc1-8e71-4427-912b-754e0820a2fd None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:02.322478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-1d541dc1-8e71-4427-912b-754e0820a2fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:02.322478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:02.322478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:02.322478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:02.322478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:02.322478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:02.322478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:02.322478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:02.322478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:02.322478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:02.322478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:02.322478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:02.322478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:02.322478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:02.322478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:02.323352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:02.323352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:02.323352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:02.323352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:02.323352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:02.323352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:02.323352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:02.323352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:02.323352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:02.323352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:02.323352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:02.323352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:02.323352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:02.323352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:02.323352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:02.323352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:02.323352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:02.324318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:02.324318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:02.324318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:02.324318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:02.324318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:02.324318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:02.324318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:02.324318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:02.328614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 265/528] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:02.329521 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d541dc1-8e71-4427-912b-754e0820a2fd None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:02.364335 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:02.378373 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:02.405882 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3e529-6b42-4eec-89dd-3d0d58baf40b None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:02.450982 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1b863-3957-40d2-b3a0-312e3190f0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:02.451600 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1b863-3957-40d2-b3a0-312e3190f0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:02.451880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1b863-3957-40d2-b3a0-312e3190f0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:02.452150 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1b863-3957-40d2-b3a0-312e3190f0a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:02.454507 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d541dc1-8e71-4427-912b-754e0820a2fd None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:02.496465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93008f-dee8-4c77-8659-98f9415f2fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:02.497386 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93008f-dee8-4c77-8659-98f9415f2fcf None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:02.497675 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93008f-dee8-4c77-8659-98f9415f2fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:02.498005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93008f-dee8-4c77-8659-98f9415f2fcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:02.515966 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93008f-dee8-4c77-8659-98f9415f2fcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:02.517633 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93008f-dee8-4c77-8659-98f9415f2fcf None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:02.537247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-de93008f-dee8-4c77-8659-98f9415f2fcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:02.537247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:02.537247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:02.537247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:02.537247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:02.537247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:02.537247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:02.537247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:02.537247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:02.537247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:02.537247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:02.537247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:02.537247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:02.537247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:02.537247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:02.538150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:02.538150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:02.538150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:02.538150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:02.538150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:02.538150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:02.538150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:02.538150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:02.538150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:02.538150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:02.538150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:02.538150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:02.538150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:02.538150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:02.538150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:02.538150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:02.538150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:02.539590 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:02.539590 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:02.539590 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:02.539590 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:02.539590 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 266/529] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:02.539590 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93008f-dee8-4c77-8659-98f9415f2fcf None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:02.555613 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1b863-3957-40d2-b3a0-312e3190f0a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:02.557112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1b863-3957-40d2-b3a0-312e3190f0a3 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:02.644450 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:02.649863 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:02.666293 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93008f-dee8-4c77-8659-98f9415f2fcf None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:02.680405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 264/530] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:23:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:23:02.680405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1b863-3957-40d2-b3a0-312e3190f0a3 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:02.717962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43696c58-7e8b-46e3-aed0-ec669f451760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:02.718977 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43696c58-7e8b-46e3-aed0-ec669f451760 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:02.719460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43696c58-7e8b-46e3-aed0-ec669f451760 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:02.719935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43696c58-7e8b-46e3-aed0-ec669f451760 None admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:02.720884 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43696c58-7e8b-46e3-aed0-ec669f451760 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:02.721426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43696c58-7e8b-46e3-aed0-ec669f451760 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:02.723560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43696c58-7e8b-46e3-aed0-ec669f451760 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:02.745133 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 267/531] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:02.746133 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43696c58-7e8b-46e3-aed0-ec669f451760 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:02.803041 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1b863-3957-40d2-b3a0-312e3190f0a3 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:02.854981 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8881e-7aa5-4cce-8821-039cafc17b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:02.857160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8881e-7aa5-4cce-8821-039cafc17b64 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:02.857901 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8881e-7aa5-4cce-8821-039cafc17b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:02.858387 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8881e-7aa5-4cce-8821-039cafc17b64 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:02.859961 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8881e-7aa5-4cce-8821-039cafc17b64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:02.860590 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8881e-7aa5-4cce-8821-039cafc17b64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:02.863097 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8881e-7aa5-4cce-8821-039cafc17b64 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:02.869470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43696c58-7e8b-46e3-aed0-ec669f451760 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:02.878988 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 265/532] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:02.880556 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8881e-7aa5-4cce-8821-039cafc17b64 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:02.916671 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b76e1b-7c9b-43c9-b134-956138b8c2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:02.917381 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b76e1b-7c9b-43c9-b134-956138b8c2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:02.918184 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b76e1b-7c9b-43c9-b134-956138b8c2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:02.918463 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b76e1b-7c9b-43c9-b134-956138b8c2a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:02.934159 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b76e1b-7c9b-43c9-b134-956138b8c2a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:02.935640 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b76e1b-7c9b-43c9-b134-956138b8c2a5 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:02.938227 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:02.941896 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:02.958737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-d6b76e1b-7c9b-43c9-b134-956138b8c2a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:02.958737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:02.958737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:02.958737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:02.958737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:02.958737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:02.958737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:02.958737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:02.958737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:02.958737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:02.958737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:02.958737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:02.958737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:02.958737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:02.958737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:02.959632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:02.959632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:02.959632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:02.959632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:02.959632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:02.959632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:02.959632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:02.959632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:02.959632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:02.959632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:02.959632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:02.959632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:02.959632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:02.959632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:02.959632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:02.959632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:02.959632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:02.960450 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:02.960450 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:02.960450 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:02.960450 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:02.964806 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 268/533] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:02.966055 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b76e1b-7c9b-43c9-b134-956138b8c2a5 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:03.014175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8881e-7aa5-4cce-8821-039cafc17b64 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:03.049670 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662395a-c740-4de8-a40b-e1b39108cadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:03.050278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662395a-c740-4de8-a40b-e1b39108cadb None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:03.050557 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662395a-c740-4de8-a40b-e1b39108cadb None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:03.050831 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662395a-c740-4de8-a40b-e1b39108cadb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:03.051620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662395a-c740-4de8-a40b-e1b39108cadb None admin] RBAC: Adding query filter params (role.id=2aa0cdfed849496db8f3692c9569d7ee, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=c50c2d1781b04a5fa8be81bce9205747) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:03.051926 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662395a-c740-4de8-a40b-e1b39108cadb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:03.053447 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662395a-c740-4de8-a40b-e1b39108cadb None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:03.059445 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 266/534] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:23:02 2022] GET /identity/v3/role_assignments?user.id=c50c2d1781b04a5fa8be81bce9205747&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=2aa0cdfed849496db8f3692c9569d7ee => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:03.060317 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662395a-c740-4de8-a40b-e1b39108cadb None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:03.079851 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b76e1b-7c9b-43c9-b134-956138b8c2a5 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:03.139527 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57d486-46ff-4478-bf28-f1223ada94a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:03.140567 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57d486-46ff-4478-bf28-f1223ada94a0 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:03.141076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57d486-46ff-4478-bf28-f1223ada94a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:03.141554 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57d486-46ff-4478-bf28-f1223ada94a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:03.152561 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57d486-46ff-4478-bf28-f1223ada94a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:03.160925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57d486-46ff-4478-bf28-f1223ada94a0 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:03.172211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662395a-c740-4de8-a40b-e1b39108cadb None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:03.180689 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:03.192810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-0e57d486-46ff-4478-bf28-f1223ada94a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:03.192810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:03.192810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:03.192810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:03.192810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:03.192810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:03.192810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:03.192810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:03.192810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:03.192810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:03.192810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:03.192810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:03.192810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:03.192810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:03.192810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:03.193751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:03.193751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:03.193751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:03.193751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:03.193751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:03.193751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:03.193751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:03.193751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:03.193751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:03.193751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:03.193751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:03.193751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:03.193751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:03.193751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:03.193751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:03.193751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:03.193751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:03.194569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:03.194569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:03.194569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:03.194569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:03.194569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:03.194569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:03.194569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:03.194569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:03.210063 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 269/535] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:03.211047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57d486-46ff-4478-bf28-f1223ada94a0 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:03.242402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d3f1f-d1a7-4b71-9c60-184b3ad43a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:03.243039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d3f1f-d1a7-4b71-9c60-184b3ad43a76 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:03.243319 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d3f1f-d1a7-4b71-9c60-184b3ad43a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:03.243593 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d3f1f-d1a7-4b71-9c60-184b3ad43a76 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:03.244300 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d3f1f-d1a7-4b71-9c60-184b3ad43a76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:03.244601 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d3f1f-d1a7-4b71-9c60-184b3ad43a76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:03.246569 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d3f1f-d1a7-4b71-9c60-184b3ad43a76 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:03.274548 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 267/536] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:03.282557 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d3f1f-d1a7-4b71-9c60-184b3ad43a76 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:03.314762 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:03.393046 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d3f1f-d1a7-4b71-9c60-184b3ad43a76 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:03.399862 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57d486-46ff-4478-bf28-f1223ada94a0 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:03.428054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95731ea0-0b65-4adb-a4ae-5dfc7e6007d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:03.428707 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95731ea0-0b65-4adb-a4ae-5dfc7e6007d1 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:03.429017 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95731ea0-0b65-4adb-a4ae-5dfc7e6007d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:03.429296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95731ea0-0b65-4adb-a4ae-5dfc7e6007d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:03.436830 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95731ea0-0b65-4adb-a4ae-5dfc7e6007d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:03.438322 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95731ea0-0b65-4adb-a4ae-5dfc7e6007d1 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:03.452419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-95731ea0-0b65-4adb-a4ae-5dfc7e6007d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:03.452419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:03.452419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:03.452419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:03.452419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:03.452419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:03.452419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:03.452419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:03.452419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:03.452419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:03.452419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:03.452419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:03.452419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:03.452419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:03.452419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:03.453324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:03.453324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:03.453324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:03.453324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:03.453324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:03.453324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:03.453324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:03.453324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:03.453324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:03.453324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:03.453324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:03.453324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:03.453324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:03.453324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:03.453324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:03.453324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:03.453324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:03.454144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:03.454144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:03.454144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:03.454144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:03.454144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:03.454144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:03.454144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:03.454144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:03.454144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 268/537] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:03.459513 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95731ea0-0b65-4adb-a4ae-5dfc7e6007d1 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:03.464256 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0355b1-e48a-493b-9c83-21d4e9e7b6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:03.465232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0355b1-e48a-493b-9c83-21d4e9e7b6aa None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:03.465878 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0355b1-e48a-493b-9c83-21d4e9e7b6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:03.466412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0355b1-e48a-493b-9c83-21d4e9e7b6aa None admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:03.467430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0355b1-e48a-493b-9c83-21d4e9e7b6aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:03.467989 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0355b1-e48a-493b-9c83-21d4e9e7b6aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:03.469869 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0355b1-e48a-493b-9c83-21d4e9e7b6aa None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:03.489260 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:03.495339 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 270/538] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:03.501762 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0355b1-e48a-493b-9c83-21d4e9e7b6aa None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:03.564275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95731ea0-0b65-4adb-a4ae-5dfc7e6007d1 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:03.638334 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a008c1e-2215-4984-986b-397310205ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:03.639318 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a008c1e-2215-4984-986b-397310205ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:03.640932 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a008c1e-2215-4984-986b-397310205ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:03.641529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a008c1e-2215-4984-986b-397310205ac4 None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:03.644141 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a008c1e-2215-4984-986b-397310205ac4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:03.644957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a008c1e-2215-4984-986b-397310205ac4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:03.647748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a008c1e-2215-4984-986b-397310205ac4 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:03.673163 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 269/539] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:03.680461 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a008c1e-2215-4984-986b-397310205ac4 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:03.716835 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0355b1-e48a-493b-9c83-21d4e9e7b6aa None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:03.734353 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:03.756109 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba874990-e4c0-47a8-a6e8-7f9d0a82cc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:03.756723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba874990-e4c0-47a8-a6e8-7f9d0a82cc34 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:03.757383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba874990-e4c0-47a8-a6e8-7f9d0a82cc34 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:03.757383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba874990-e4c0-47a8-a6e8-7f9d0a82cc34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:03.758076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba874990-e4c0-47a8-a6e8-7f9d0a82cc34 None admin] RBAC: Adding query filter params (role.id=2aa0cdfed849496db8f3692c9569d7ee, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=dd352d423bff418f932b434c51b15d23) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:03.758381 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba874990-e4c0-47a8-a6e8-7f9d0a82cc34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:03.759824 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba874990-e4c0-47a8-a6e8-7f9d0a82cc34 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:03.765746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 271/540] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:23:03 2022] GET /identity/v3/role_assignments?user.id=dd352d423bff418f932b434c51b15d23&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=2aa0cdfed849496db8f3692c9569d7ee => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:03.766916 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3fd34a-3198-44d2-a365-979193ac3d51 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:03.767196 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3fd34a-3198-44d2-a365-979193ac3d51 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:03.767461 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3fd34a-3198-44d2-a365-979193ac3d51 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:03.777474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 272/541] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:03.791198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71450978-44e6-4815-af66-bc8b709c742d None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:03.791584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71450978-44e6-4815-af66-bc8b709c742d None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:03.791751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71450978-44e6-4815-af66-bc8b709c742d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:03.803630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:03.820482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a008c1e-2215-4984-986b-397310205ac4 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:03.834714 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-71450978-44e6-4815-af66-bc8b709c742d None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:03.835922 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-71450978-44e6-4815-af66-bc8b709c742d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:03.872006 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65cb3a-ce16-4838-a8bd-8ed3a76124f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:03.873245 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65cb3a-ce16-4838-a8bd-8ed3a76124f3 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:03.873750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65cb3a-ce16-4838-a8bd-8ed3a76124f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:03.874221 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65cb3a-ce16-4838-a8bd-8ed3a76124f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:03.875328 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65cb3a-ce16-4838-a8bd-8ed3a76124f3 None admin] RBAC: Adding query filter params (role.id=2aa0cdfed849496db8f3692c9569d7ee, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=193dd9fee1844275ad0dab602e93df11) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:03.875918 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65cb3a-ce16-4838-a8bd-8ed3a76124f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:03.877698 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65cb3a-ce16-4838-a8bd-8ed3a76124f3 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:03.891745 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 270/542] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:23:03 2022] GET /identity/v3/role_assignments?user.id=193dd9fee1844275ad0dab602e93df11&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=2aa0cdfed849496db8f3692c9569d7ee => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:03.922381 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 273/543] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:23:03.935675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187081a0-8f8d-444b-8889-a7c6dd0d8459 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:03.935990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187081a0-8f8d-444b-8889-a7c6dd0d8459 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:03.936228 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187081a0-8f8d-444b-8889-a7c6dd0d8459 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:03.938026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 271/544] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:03.944170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd987f6a-a29d-4192-b759-15b3edb44b16 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:03.944495 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd987f6a-a29d-4192-b759-15b3edb44b16 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:03.944733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd987f6a-a29d-4192-b759-15b3edb44b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:03.955834 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187081a0-8f8d-444b-8889-a7c6dd0d8459 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:03.969671 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-cd987f6a-a29d-4192-b759-15b3edb44b16 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:03.971126 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-cd987f6a-a29d-4192-b759-15b3edb44b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:04.048142 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 274/545] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:23:04.083777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd987f6a-a29d-4192-b759-15b3edb44b16 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:04.123159 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187081a0-8f8d-444b-8889-a7c6dd0d8459 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:04.127732 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:04.191492 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd987f6a-a29d-4192-b759-15b3edb44b16 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:04.202505 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4319a6b-fa8f-43ce-b842-202a40b0d141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:04.203272 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4319a6b-fa8f-43ce-b842-202a40b0d141 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:04.203561 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4319a6b-fa8f-43ce-b842-202a40b0d141 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:04.203838 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4319a6b-fa8f-43ce-b842-202a40b0d141 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:04.227926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5cd2f-f014-4d43-a932-a63b9a9813b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:04.228814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5cd2f-f014-4d43-a932-a63b9a9813b2 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:04.229206 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5cd2f-f014-4d43-a932-a63b9a9813b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:04.229548 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5cd2f-f014-4d43-a932-a63b9a9813b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:04.233214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4319a6b-fa8f-43ce-b842-202a40b0d141 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:04.239054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-c4319a6b-fa8f-43ce-b842-202a40b0d141 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:04.239054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:04.239054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:04.239054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:04.239054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:04.239054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:04.239054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:04.239054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:04.239054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:04.239054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:04.239054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:04.239054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:04.239054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:04.239054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:04.239054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:04.239054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:04.239934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:04.239934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:04.239934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:04.239934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:04.239934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:04.239934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:04.239934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:04.239934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:04.239934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:04.239934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:04.239934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:04.239934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:04.239934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:04.239934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:04.239934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:04.239934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:04.240702 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:04.240702 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:04.240702 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:04.240702 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:04.240702 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:04.240702 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:04.240702 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:04.240702 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:04.240702 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:04.240702 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:04.240702 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:04.240702 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:04.240702 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 272/546] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:23:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:23:04.241386 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5cd2f-f014-4d43-a932-a63b9a9813b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:04.247032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-b2a5cd2f-f014-4d43-a932-a63b9a9813b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:04.247032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:04.247032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:04.247032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:04.247032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:04.247032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:04.247032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:04.247032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:04.247032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:04.247032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:04.247032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:04.247032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:04.247032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:04.247032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:04.247032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:04.247032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:04.248847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:04.248847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:04.248847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:04.248847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:04.248847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:04.248847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:04.248847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:04.248847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:04.248847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:04.248847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:04.248847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:04.248847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:04.248847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:04.248847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:04.248847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:04.248847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:04.250117 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:04.250117 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:04.250117 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:04.250117 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:04.250117 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:04.250117 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:04.250117 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:04.250117 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:04.250117 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:04.250117 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:04.250117 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:04.250117 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:04.250117 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 275/547] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:23:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:23:04.253876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4319a6b-fa8f-43ce-b842-202a40b0d141 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:04.254776 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5cd2f-f014-4d43-a932-a63b9a9813b2 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:04.283742 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:04.361286 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:04.365427 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5cd2f-f014-4d43-a932-a63b9a9813b2 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:04.404414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524e467-2910-41ab-955b-cc15c58304cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:04.405212 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524e467-2910-41ab-955b-cc15c58304cf None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:04.405493 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524e467-2910-41ab-955b-cc15c58304cf None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:04.405766 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524e467-2910-41ab-955b-cc15c58304cf None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:04.406489 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524e467-2910-41ab-955b-cc15c58304cf None admin] RBAC: Adding query filter params (name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:04.406792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524e467-2910-41ab-955b-cc15c58304cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:04.408237 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524e467-2910-41ab-955b-cc15c58304cf None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:04.417396 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 276/548] 10.222.0.74 () {56 vars in 1103 bytes} [Wed Jul 27 16:23:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:04.423433 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524e467-2910-41ab-955b-cc15c58304cf None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:04.502109 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4319a6b-fa8f-43ce-b842-202a40b0d141 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:04.570747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524e467-2910-41ab-955b-cc15c58304cf None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:04.610267 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e289786-2b76-41a0-bc16-a1745709249c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:04.610894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e289786-2b76-41a0-bc16-a1745709249c None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:04.611179 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e289786-2b76-41a0-bc16-a1745709249c None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:04.611450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e289786-2b76-41a0-bc16-a1745709249c None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:04.612170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e289786-2b76-41a0-bc16-a1745709249c None admin] RBAC: Adding query filter params (name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:04.612472 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e289786-2b76-41a0-bc16-a1745709249c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:04.614081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e289786-2b76-41a0-bc16-a1745709249c None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:04.616727 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd35d9-7166-444a-b621-654bc89aa59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:04.617396 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd35d9-7166-444a-b621-654bc89aa59c None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:04.617674 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd35d9-7166-444a-b621-654bc89aa59c None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:04.617944 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd35d9-7166-444a-b621-654bc89aa59c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:04.622791 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 273/549] 10.222.0.74 () {56 vars in 1103 bytes} [Wed Jul 27 16:23:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:04.629894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e289786-2b76-41a0-bc16-a1745709249c None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:04.631550 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd35d9-7166-444a-b621-654bc89aa59c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:04.633144 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd35d9-7166-444a-b621-654bc89aa59c None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:04.659771 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-08fd35d9-7166-444a-b621-654bc89aa59c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:04.659771 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:04.659771 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:04.659771 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:04.659771 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:04.659771 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:04.659771 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:04.659771 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:04.659771 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:04.659771 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:04.659771 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:04.659771 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:04.659771 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:04.659771 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:04.659771 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:04.662383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:04.662383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:04.662383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:04.662383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:04.662383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:04.662383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:04.662383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:04.662383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:04.662383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:04.662383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:04.662383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:04.662383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:04.662383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:04.662383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:04.662383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:04.662383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:04.662383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:04.664638 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:04.664638 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:04.664638 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:04.664638 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:04.668386 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 277/550] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:04.673723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd35d9-7166-444a-b621-654bc89aa59c None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:04.698995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:04.703451 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:04.736874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e289786-2b76-41a0-bc16-a1745709249c None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:04.787693 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461a77d-4534-4c23-94f3-d618542a0172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:04.788739 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461a77d-4534-4c23-94f3-d618542a0172 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:04.790320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461a77d-4534-4c23-94f3-d618542a0172 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:04.791317 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461a77d-4534-4c23-94f3-d618542a0172 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:04.810712 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461a77d-4534-4c23-94f3-d618542a0172 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:04.816958 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461a77d-4534-4c23-94f3-d618542a0172 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:04.838880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-0461a77d-4534-4c23-94f3-d618542a0172 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:04.838880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:04.838880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:04.838880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:04.838880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:04.838880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:04.838880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:04.838880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:04.838880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:04.838880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:04.838880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:04.838880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:04.838880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:04.838880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:04.838880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:04.840484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:04.840484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:04.840484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:04.840484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:04.840484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:04.840484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:04.840484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:04.840484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:04.840484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:04.840484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:04.840484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:04.840484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:04.840484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:04.840484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:04.840484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:04.840484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:04.840484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:04.841469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:04.841469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:04.841469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:04.841469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:04.850164 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 274/551] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:04.854171 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461a77d-4534-4c23-94f3-d618542a0172 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:04.874079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd35d9-7166-444a-b621-654bc89aa59c None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:04.913549 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554a7aa-b7e0-484e-8621-9abb13c246fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:04.914691 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554a7aa-b7e0-484e-8621-9abb13c246fd None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:04.915132 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554a7aa-b7e0-484e-8621-9abb13c246fd None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:04.915551 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554a7aa-b7e0-484e-8621-9abb13c246fd None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:04.916385 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554a7aa-b7e0-484e-8621-9abb13c246fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:04.916850 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554a7aa-b7e0-484e-8621-9abb13c246fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:04.919468 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554a7aa-b7e0-484e-8621-9abb13c246fd None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:04.932925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 278/552] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:04.945858 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554a7aa-b7e0-484e-8621-9abb13c246fd None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:05.020732 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461a77d-4534-4c23-94f3-d618542a0172 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:05.047013 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:05.056381 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f765e33-ab01-4bf1-9244-3694d5ba0481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:05.057194 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f765e33-ab01-4bf1-9244-3694d5ba0481 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:05.057480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f765e33-ab01-4bf1-9244-3694d5ba0481 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:05.057757 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f765e33-ab01-4bf1-9244-3694d5ba0481 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:05.058512 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f765e33-ab01-4bf1-9244-3694d5ba0481 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:05.058821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f765e33-ab01-4bf1-9244-3694d5ba0481 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:05.060314 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f765e33-ab01-4bf1-9244-3694d5ba0481 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:05.073002 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 275/553] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:05.079305 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f765e33-ab01-4bf1-9244-3694d5ba0481 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:05.127335 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:05.201206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f765e33-ab01-4bf1-9244-3694d5ba0481 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:05.236487 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554a7aa-b7e0-484e-8621-9abb13c246fd None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:05.243170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb350c-cd0a-487f-90b6-19cfc38208c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:05.249214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb350c-cd0a-487f-90b6-19cfc38208c9 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:05.249731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb350c-cd0a-487f-90b6-19cfc38208c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:05.250230 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb350c-cd0a-487f-90b6-19cfc38208c9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:05.261868 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb350c-cd0a-487f-90b6-19cfc38208c9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:05.264280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb350c-cd0a-487f-90b6-19cfc38208c9 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:05.286620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-2fcb350c-cd0a-487f-90b6-19cfc38208c9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 16:23:05.286620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:05.286620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:05.286620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:05.286620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:05.286620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:05.286620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:05.286620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:05.286620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:05.286620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:05.286620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:05.286620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:05.286620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:05.286620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:05.286620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:05.286620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:05.287634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:05.287634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:05.287634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:05.287634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:05.287634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:05.287634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:05.287634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:05.287634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:05.287634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:05.287634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:05.287634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:05.287634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:05.287634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:05.287634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:05.287634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:05.287634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:05.287634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:05.287634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:05.288529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:05.288529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:05.288529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:05.288529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:05.288529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:05.288529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:05.288529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:05.288529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:05.288529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 16:23:05.288529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:05.288529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 276/554] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:23:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:05.294616 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb350c-cd0a-487f-90b6-19cfc38208c9 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:05.301125 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04e1b3-e402-4ae1-ad67-375d0d31a23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:05.303018 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04e1b3-e402-4ae1-ad67-375d0d31a23d None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:05.303518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04e1b3-e402-4ae1-ad67-375d0d31a23d None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:05.304013 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04e1b3-e402-4ae1-ad67-375d0d31a23d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:05.313147 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c04e1b3-e402-4ae1-ad67-375d0d31a23d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:05.315545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c04e1b3-e402-4ae1-ad67-375d0d31a23d None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:05.329361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-6c04e1b3-e402-4ae1-ad67-375d0d31a23d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 16:23:05.329361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:05.329361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:05.329361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:05.329361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:05.329361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:05.329361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:05.329361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:05.329361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:05.329361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:05.329361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:05.329361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:05.329361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:05.329361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:05.329361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:05.329361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:05.330295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:05.330295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:05.330295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:05.330295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:05.330295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:05.330295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:05.330295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:05.330295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:05.330295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:05.330295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:05.330295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:05.330295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:05.330295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:05.330295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:05.330295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:05.330295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:05.330295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:05.330295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:05.331658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:05.331658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:05.331658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:05.331658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:05.331658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:05.331658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:05.331658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:05.331658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:05.331658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 16:23:05.331658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:05.334636 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 279/555] 10.222.0.74 () {56 vars in 1093 bytes} [Wed Jul 27 16:23:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:05.340542 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04e1b3-e402-4ae1-ad67-375d0d31a23d None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:05.393240 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:05.484793 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04e1b3-e402-4ae1-ad67-375d0d31a23d None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:05.486047 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb350c-cd0a-487f-90b6-19cfc38208c9 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:05.516582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:05.522370 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049705ab-6ac0-4e99-a1af-926bf34aa184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:05.522986 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049705ab-6ac0-4e99-a1af-926bf34aa184 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:05.523272 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049705ab-6ac0-4e99-a1af-926bf34aa184 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:05.523545 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049705ab-6ac0-4e99-a1af-926bf34aa184 None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:05.524140 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5319ae3-310c-47f0-9c6e-521a09733590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:05.524351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049705ab-6ac0-4e99-a1af-926bf34aa184 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:05.524629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049705ab-6ac0-4e99-a1af-926bf34aa184 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:05.525421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5319ae3-310c-47f0-9c6e-521a09733590 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:05.526010 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5319ae3-310c-47f0-9c6e-521a09733590 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:05.526115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049705ab-6ac0-4e99-a1af-926bf34aa184 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:05.526603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5319ae3-310c-47f0-9c6e-521a09733590 None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:05.528376 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5319ae3-310c-47f0-9c6e-521a09733590 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:05.528883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5319ae3-310c-47f0-9c6e-521a09733590 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:05.531370 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5319ae3-310c-47f0-9c6e-521a09733590 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:05.545283 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 277/556] 10.222.0.74 () {56 vars in 1135 bytes} [Wed Jul 27 16:23:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:05.546529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25539ee-edc1-40a3-8ce6-5aa0926cc203 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:05.546815 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25539ee-edc1-40a3-8ce6-5aa0926cc203 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:05.547084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25539ee-edc1-40a3-8ce6-5aa0926cc203 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:05.552298 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 278/557] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:05.552298 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25539ee-edc1-40a3-8ce6-5aa0926cc203 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:05.555165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 280/558] 10.222.0.74 () {56 vars in 1133 bytes} [Wed Jul 27 16:23:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:05.555726 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d61d04-02f8-469e-9f29-09a2d5da2d1a None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:05.556234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d61d04-02f8-469e-9f29-09a2d5da2d1a None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:05.556716 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d61d04-02f8-469e-9f29-09a2d5da2d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:05.575133 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-e6d61d04-02f8-469e-9f29-09a2d5da2d1a None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:05.576578 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e6d61d04-02f8-469e-9f29-09a2d5da2d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:05.653463 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 281/559] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:23:05.654338 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d61d04-02f8-469e-9f29-09a2d5da2d1a None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:05.673610 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25539ee-edc1-40a3-8ce6-5aa0926cc203 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:05.675801 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:05.716683 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509792f4-9a20-4660-8add-80997b6b6d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:05.718116 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509792f4-9a20-4660-8add-80997b6b6d08 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:05.718809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509792f4-9a20-4660-8add-80997b6b6d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:05.719236 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509792f4-9a20-4660-8add-80997b6b6d08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:05.774807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d61d04-02f8-469e-9f29-09a2d5da2d1a None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:05.797931 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:05.813619 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ddb58-224e-4062-bf44-5e0a37f7c01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:05.814443 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ddb58-224e-4062-bf44-5e0a37f7c01c None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:05.814753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509792f4-9a20-4660-8add-80997b6b6d08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:05.814872 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ddb58-224e-4062-bf44-5e0a37f7c01c None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:05.815260 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ddb58-224e-4062-bf44-5e0a37f7c01c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:05.816686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509792f4-9a20-4660-8add-80997b6b6d08 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:05.829217 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ddb58-224e-4062-bf44-5e0a37f7c01c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:05.830842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ddb58-224e-4062-bf44-5e0a37f7c01c None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:05.847030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-f13ddb58-224e-4062-bf44-5e0a37f7c01c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:05.847030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:05.847030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:05.847030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:05.847030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:05.847030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:05.847030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:05.847030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:05.847030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:05.847030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:05.847030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:05.847030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:05.847030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:05.847030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:05.847030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:05.848025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:05.848025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:05.848025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:05.848025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:05.848025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:05.848025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:05.848025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:05.848025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:05.848025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:05.848025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:05.848025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:05.848025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:05.848025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:05.848025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:05.848025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:05.848025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:05.848025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:05.848883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:05.848883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:05.848883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:05.848883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:05.853414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 282/560] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:05.854350 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ddb58-224e-4062-bf44-5e0a37f7c01c None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:05.941345 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 279/561] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:23:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:23:05.941345 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509792f4-9a20-4660-8add-80997b6b6d08 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:05.941345 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:05.983807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ddb58-224e-4062-bf44-5e0a37f7c01c None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:06.021761 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814f4b0-ca00-464f-8e40-356ed2638a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:06.022770 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814f4b0-ca00-464f-8e40-356ed2638a71 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:06.023255 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814f4b0-ca00-464f-8e40-356ed2638a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:06.023730 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814f4b0-ca00-464f-8e40-356ed2638a71 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:06.035831 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814f4b0-ca00-464f-8e40-356ed2638a71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:06.041518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-f814f4b0-ca00-464f-8e40-356ed2638a71 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:06.041518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:06.041518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:06.041518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:06.041518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:06.041518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:06.041518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:06.041518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:06.041518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:06.041518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:06.041518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:06.041518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:06.041518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:06.041518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:06.041518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:06.041518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:06.042416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:06.042416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:06.042416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:06.042416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:06.042416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:06.042416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:06.042416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:06.042416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:06.042416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:06.042416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:06.042416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:06.042416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:06.042416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:06.042416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:06.042416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:06.042416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:06.043877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:06.043877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:06.043877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:06.043877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:06.043877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:06.043877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:06.043877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:06.043877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:06.043877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:06.043877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:06.043877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:06.043877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:06.046613 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 283/562] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:23:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:06.047725 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814f4b0-ca00-464f-8e40-356ed2638a71 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:06.074903 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509792f4-9a20-4660-8add-80997b6b6d08 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:06.088407 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:06.113213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0246a2-eb40-4fb0-aba0-79c07372480a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:06.113842 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0246a2-eb40-4fb0-aba0-79c07372480a None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:06.114123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0246a2-eb40-4fb0-aba0-79c07372480a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:06.114457 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0246a2-eb40-4fb0-aba0-79c07372480a None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:06.115150 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0246a2-eb40-4fb0-aba0-79c07372480a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:06.115500 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0246a2-eb40-4fb0-aba0-79c07372480a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:06.117166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0246a2-eb40-4fb0-aba0-79c07372480a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:06.130070 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 280/563] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:06.131008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0246a2-eb40-4fb0-aba0-79c07372480a None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:06.192791 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814f4b0-ca00-464f-8e40-356ed2638a71 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:06.211792 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:06.236631 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f728aee-391a-43c3-b26e-5b0faf5dcca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:06.237314 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f728aee-391a-43c3-b26e-5b0faf5dcca3 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:06.237606 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f728aee-391a-43c3-b26e-5b0faf5dcca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:06.237882 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f728aee-391a-43c3-b26e-5b0faf5dcca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:06.249440 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f728aee-391a-43c3-b26e-5b0faf5dcca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:06.250887 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f728aee-391a-43c3-b26e-5b0faf5dcca3 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:06.254374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0246a2-eb40-4fb0-aba0-79c07372480a None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:06.265363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-9f728aee-391a-43c3-b26e-5b0faf5dcca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:06.265363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:06.265363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:06.265363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:06.265363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:06.265363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:06.265363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:06.265363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:06.265363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:06.265363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:06.265363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:06.265363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:06.265363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:06.265363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:06.265363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:06.266408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:06.266408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:06.266408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:06.266408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:06.266408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:06.266408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:06.266408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:06.266408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:06.266408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:06.266408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:06.266408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:06.266408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:06.266408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:06.266408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:06.266408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:06.266408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:06.266408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:06.267630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:06.267630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:06.267630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:06.267630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:06.267630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 284/564] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:06.267630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f728aee-391a-43c3-b26e-5b0faf5dcca3 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:06.290760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970995c1-6275-4870-ad5a-384a93a2c231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:06.291311 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970995c1-6275-4870-ad5a-384a93a2c231 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:06.291590 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970995c1-6275-4870-ad5a-384a93a2c231 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:06.291866 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970995c1-6275-4870-ad5a-384a93a2c231 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:06.292606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970995c1-6275-4870-ad5a-384a93a2c231 None admin] RBAC: Adding query filter params (name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:06.292950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970995c1-6275-4870-ad5a-384a93a2c231 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:06.294328 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970995c1-6275-4870-ad5a-384a93a2c231 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:06.303206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 281/565] 10.222.0.74 () {56 vars in 1103 bytes} [Wed Jul 27 16:23:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:06.306395 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ff493-c24d-404c-9883-89f63c880218 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:06.306722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ff493-c24d-404c-9883-89f63c880218 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:06.306997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ff493-c24d-404c-9883-89f63c880218 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:06.308923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 282/566] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:06.310141 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566a11c-e6c2-4ebc-bb87-784e8e743250 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:06.310449 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566a11c-e6c2-4ebc-bb87-784e8e743250 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:06.310917 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566a11c-e6c2-4ebc-bb87-784e8e743250 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:06.312225 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 283/567] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:06.313002 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566a11c-e6c2-4ebc-bb87-784e8e743250 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:06.358887 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:06.375233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f728aee-391a-43c3-b26e-5b0faf5dcca3 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:06.408162 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33b7b8-9651-48dc-9939-fbe84a12b7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:06.408802 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33b7b8-9651-48dc-9939-fbe84a12b7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:06.409113 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33b7b8-9651-48dc-9939-fbe84a12b7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:06.409391 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33b7b8-9651-48dc-9939-fbe84a12b7d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:06.413921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566a11c-e6c2-4ebc-bb87-784e8e743250 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:06.416019 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33b7b8-9651-48dc-9939-fbe84a12b7d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:06.418024 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33b7b8-9651-48dc-9939-fbe84a12b7d3 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:06.429406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-1f33b7b8-9651-48dc-9939-fbe84a12b7d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:06.429406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:06.429406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:06.429406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:06.429406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:06.429406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:06.429406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:06.429406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:06.429406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:06.429406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:06.429406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:06.429406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:06.429406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:06.429406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:06.429406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:06.430288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:06.430288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:06.430288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:06.430288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:06.430288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:06.430288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:06.430288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:06.430288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:06.430288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:06.430288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:06.430288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:06.430288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:06.430288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:06.430288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:06.430288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:06.430288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:06.430288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:06.431167 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:06.431167 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:06.431167 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:06.431167 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:06.431167 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:06.431167 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:06.431167 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:06.431167 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:06.431167 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 285/568] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:06.431648 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33b7b8-9651-48dc-9939-fbe84a12b7d3 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:06.436720 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:06.447809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf474a-afd2-40d2-bf5e-600407ee7501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:06.448419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf474a-afd2-40d2-bf5e-600407ee7501 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:06.448698 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf474a-afd2-40d2-bf5e-600407ee7501 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:06.449506 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf474a-afd2-40d2-bf5e-600407ee7501 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:06.450435 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf474a-afd2-40d2-bf5e-600407ee7501 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:06.450742 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf474a-afd2-40d2-bf5e-600407ee7501 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:06.452117 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf474a-afd2-40d2-bf5e-600407ee7501 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:06.464211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 284/569] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:06.465863 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10bbdf-f67a-4681-8be4-69702e6a8b26 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:06.466144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10bbdf-f67a-4681-8be4-69702e6a8b26 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:06.466416 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10bbdf-f67a-4681-8be4-69702e6a8b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:06.483176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-6b10bbdf-f67a-4681-8be4-69702e6a8b26 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:06.484385 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6b10bbdf-f67a-4681-8be4-69702e6a8b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:06.540248 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33b7b8-9651-48dc-9939-fbe84a12b7d3 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:06.549371 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 285/570] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:23:06.550668 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcd633-dee4-4bfb-9317-30c05586197c None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:06.550964 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcd633-dee4-4bfb-9317-30c05586197c None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:06.551245 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcd633-dee4-4bfb-9317-30c05586197c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:06.571100 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-c9bcd633-dee4-4bfb-9317-30c05586197c None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:06.572391 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c9bcd633-dee4-4bfb-9317-30c05586197c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:06.580554 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de9c98-4bc5-41e5-8f51-1983f95b42f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:06.581240 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de9c98-4bc5-41e5-8f51-1983f95b42f1 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:06.581523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de9c98-4bc5-41e5-8f51-1983f95b42f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:06.581798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de9c98-4bc5-41e5-8f51-1983f95b42f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:06.593468 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de9c98-4bc5-41e5-8f51-1983f95b42f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:06.594954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de9c98-4bc5-41e5-8f51-1983f95b42f1 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:06.597655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:06.609847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-a2de9c98-4bc5-41e5-8f51-1983f95b42f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:06.609847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:06.609847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:06.609847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:06.609847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:06.609847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:06.609847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:06.609847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:06.609847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:06.609847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:06.609847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:06.609847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:06.609847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:06.609847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:06.609847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:06.610945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:06.610945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:06.610945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:06.610945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:06.610945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:06.610945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:06.610945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:06.610945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:06.610945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:06.610945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:06.610945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:06.610945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:06.610945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:06.610945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:06.610945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:06.610945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:06.610945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:06.611919 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:06.611919 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:06.611919 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:06.611919 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:06.611919 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 286/571] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:06.611919 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de9c98-4bc5-41e5-8f51-1983f95b42f1 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:06.639815 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 286/572] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:23:06.649236 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcd633-dee4-4bfb-9317-30c05586197c None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:06.685058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:06.724423 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de9c98-4bc5-41e5-8f51-1983f95b42f1 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:06.738598 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcd633-dee4-4bfb-9317-30c05586197c None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:06.758347 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179fab6c-24e8-44a9-9fc7-d83e630f64f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:06.758961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179fab6c-24e8-44a9-9fc7-d83e630f64f8 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:06.759244 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179fab6c-24e8-44a9-9fc7-d83e630f64f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:06.759517 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179fab6c-24e8-44a9-9fc7-d83e630f64f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:06.760255 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179fab6c-24e8-44a9-9fc7-d83e630f64f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:06.760558 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179fab6c-24e8-44a9-9fc7-d83e630f64f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:06.762608 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179fab6c-24e8-44a9-9fc7-d83e630f64f8 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:06.771804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13a71c-281b-4f28-a2a4-4bf1720f9192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:06.772433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13a71c-281b-4f28-a2a4-4bf1720f9192 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:06.772714 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13a71c-281b-4f28-a2a4-4bf1720f9192 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:06.773017 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13a71c-281b-4f28-a2a4-4bf1720f9192 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:06.779842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 287/573] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:06.779842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179fab6c-24e8-44a9-9fc7-d83e630f64f8 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:06.780419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13a71c-281b-4f28-a2a4-4bf1720f9192 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:06.781893 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13a71c-281b-4f28-a2a4-4bf1720f9192 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:06.793152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-ea13a71c-281b-4f28-a2a4-4bf1720f9192 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:06.793152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:06.793152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:06.793152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:06.793152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:06.793152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:06.793152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:06.793152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:06.793152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:06.793152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:06.793152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:06.793152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:06.793152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:06.793152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:06.793152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:06.794066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:06.794066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:06.794066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:06.794066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:06.794066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:06.794066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:06.794066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:06.794066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:06.794066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:06.794066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:06.794066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:06.794066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:06.794066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:06.794066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:06.794066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:06.794066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:06.794066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:06.794936 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:06.794936 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:06.794936 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:06.794936 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:06.794936 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:06.794936 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:06.794936 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:06.794936 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:06.794936 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 287/574] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:06.795404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13a71c-281b-4f28-a2a4-4bf1720f9192 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:06.833543 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:06.877474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179fab6c-24e8-44a9-9fc7-d83e630f64f8 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:06.892532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13a71c-281b-4f28-a2a4-4bf1720f9192 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:06.894830 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:06.910635 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a484d61-1709-49d2-ba15-6e2b201b7273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:06.911275 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a484d61-1709-49d2-ba15-6e2b201b7273 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:06.911551 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a484d61-1709-49d2-ba15-6e2b201b7273 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:06.911826 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a484d61-1709-49d2-ba15-6e2b201b7273 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:06.921844 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a484d61-1709-49d2-ba15-6e2b201b7273 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:06.926634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36f11d-23fb-46a4-9d40-6d2ce35886b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:06.926752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-4a484d61-1709-49d2-ba15-6e2b201b7273 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:06.926752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:06.926752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:06.926752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:06.926752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:06.926752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:06.926752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:06.926752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:06.926752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:06.926752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:06.926752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:06.926752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:06.926752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:06.926752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:06.926752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:06.926752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:06.927610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:06.927610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:06.927610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:06.927610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:06.927610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:06.927610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:06.927610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:06.927610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:06.927610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:06.927610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:06.927610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:06.927610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:06.927610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:06.927610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:06.927610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:06.927610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:06.928439 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:06.928439 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:06.928439 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:06.928439 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:06.928439 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:06.928439 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:06.928439 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:06.928439 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:06.928439 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:06.928439 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:06.928439 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:06.928439 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:06.929089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36f11d-23fb-46a4-9d40-6d2ce35886b5 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:06.929089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36f11d-23fb-46a4-9d40-6d2ce35886b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:06.929089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36f11d-23fb-46a4-9d40-6d2ce35886b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:06.929279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 288/575] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:23:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:06.929347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36f11d-23fb-46a4-9d40-6d2ce35886b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:06.929347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36f11d-23fb-46a4-9d40-6d2ce35886b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:06.929475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a484d61-1709-49d2-ba15-6e2b201b7273 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:06.930158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36f11d-23fb-46a4-9d40-6d2ce35886b5 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:06.942200 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 288/576] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:06.942977 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36f11d-23fb-46a4-9d40-6d2ce35886b5 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:07.035155 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a484d61-1709-49d2-ba15-6e2b201b7273 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:07.038445 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36f11d-23fb-46a4-9d40-6d2ce35886b5 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:07.051836 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:07.070071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68b92c-6ecd-404e-9bc1-88f2471fff01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:07.070739 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68b92c-6ecd-404e-9bc1-88f2471fff01 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:07.071020 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68b92c-6ecd-404e-9bc1-88f2471fff01 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:07.071296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68b92c-6ecd-404e-9bc1-88f2471fff01 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:07.073088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592ff33-0be1-4e23-99db-237efc59e53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:07.073744 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592ff33-0be1-4e23-99db-237efc59e53d None admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:07.074092 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592ff33-0be1-4e23-99db-237efc59e53d None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:07.074360 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592ff33-0be1-4e23-99db-237efc59e53d None admin] PATH_INFO: `/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/56dfafd15e284b7986ba07dd3819bf65/roles/2aa0cdfed849496db8f3692c9569d7ee` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:07.081471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68b92c-6ecd-404e-9bc1-88f2471fff01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:07.086419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-dc68b92c-6ecd-404e-9bc1-88f2471fff01 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:07.086419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:07.086419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:07.086419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:07.086419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:07.086419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:07.086419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:07.086419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:07.086419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:07.086419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:07.086419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:07.086419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:07.086419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:07.086419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:07.086419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:07.086419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:07.087281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:07.087281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:07.087281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:07.087281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:07.087281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:07.087281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:07.087281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:07.087281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:07.087281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:07.087281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:07.087281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:07.087281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:07.087281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:07.087281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:07.087281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:07.087281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:07.088101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:07.088101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:07.088101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:07.088101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:07.088101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:07.088101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:07.088101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:07.088101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:07.088101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:07.088101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:07.088101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:07.088101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:07.088101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 289/577] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:23:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:07.088769 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68b92c-6ecd-404e-9bc1-88f2471fff01 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:07.093759 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592ff33-0be1-4e23-99db-237efc59e53d None admin] RBAC: Authorizing `identity:create_grant(project_id=8d6bcdd09cf24a5c8003951db37020cf, user_id=56dfafd15e284b7986ba07dd3819bf65, role_id=2aa0cdfed849496db8f3692c9569d7ee)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:07.095190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592ff33-0be1-4e23-99db-237efc59e53d None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:07.107713 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 289/578] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:23:06 2022] PUT /identity/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/56dfafd15e284b7986ba07dd3819bf65/roles/2aa0cdfed849496db8f3692c9569d7ee => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:23:07.108492 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592ff33-0be1-4e23-99db-237efc59e53d None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:07.126221 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:07.223867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68b92c-6ecd-404e-9bc1-88f2471fff01 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:07.232826 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592ff33-0be1-4e23-99db-237efc59e53d None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:07.268762 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd86fd-878e-49f9-8f95-ecc7e78f0202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:07.269435 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd86fd-878e-49f9-8f95-ecc7e78f0202 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:07.269718 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd86fd-878e-49f9-8f95-ecc7e78f0202 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:07.269987 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd86fd-878e-49f9-8f95-ecc7e78f0202 None admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:07.270727 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd86fd-878e-49f9-8f95-ecc7e78f0202 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:07.271040 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd86fd-878e-49f9-8f95-ecc7e78f0202 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:07.272449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd86fd-878e-49f9-8f95-ecc7e78f0202 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:07.276817 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a7448-2210-4d26-8c9b-4f9b839f6d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:07.277505 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a7448-2210-4d26-8c9b-4f9b839f6d99 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:07.277788 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a7448-2210-4d26-8c9b-4f9b839f6d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:07.278064 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a7448-2210-4d26-8c9b-4f9b839f6d99 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:07.278783 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a7448-2210-4d26-8c9b-4f9b839f6d99 None admin] RBAC: Adding query filter params (name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:07.279095 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a7448-2210-4d26-8c9b-4f9b839f6d99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:07.280461 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a7448-2210-4d26-8c9b-4f9b839f6d99 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:07.285471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 290/579] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:07.286733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd86fd-878e-49f9-8f95-ecc7e78f0202 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:07.289240 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 290/580] 10.222.0.74 () {56 vars in 1103 bytes} [Wed Jul 27 16:23:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:07.290077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a7448-2210-4d26-8c9b-4f9b839f6d99 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:07.329546 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:07.374080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:07.399116 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd86fd-878e-49f9-8f95-ecc7e78f0202 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:07.402671 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a7448-2210-4d26-8c9b-4f9b839f6d99 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:07.435435 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf47bf9-773a-4795-a95c-fb95d9e9989a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:07.436080 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf47bf9-773a-4795-a95c-fb95d9e9989a None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:07.436361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf47bf9-773a-4795-a95c-fb95d9e9989a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:07.436640 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf47bf9-773a-4795-a95c-fb95d9e9989a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:07.439863 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80a314-4e53-41a2-b96b-f662539d8faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:07.440485 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80a314-4e53-41a2-b96b-f662539d8faf None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:07.440766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80a314-4e53-41a2-b96b-f662539d8faf None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:07.441070 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80a314-4e53-41a2-b96b-f662539d8faf None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:07.441803 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80a314-4e53-41a2-b96b-f662539d8faf None admin] RBAC: Adding query filter params (name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:07.442085 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80a314-4e53-41a2-b96b-f662539d8faf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:07.443459 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80a314-4e53-41a2-b96b-f662539d8faf None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:07.444356 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf47bf9-773a-4795-a95c-fb95d9e9989a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:07.445844 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf47bf9-773a-4795-a95c-fb95d9e9989a None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:07.457387 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 291/581] 10.222.0.74 () {56 vars in 1103 bytes} [Wed Jul 27 16:23:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:07.457387 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80a314-4e53-41a2-b96b-f662539d8faf None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:07.463449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-ccf47bf9-773a-4795-a95c-fb95d9e9989a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 16:23:07.463449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:07.463449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:07.463449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:07.463449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:07.463449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:07.463449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:07.463449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:07.463449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:07.463449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:07.463449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:07.463449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:07.463449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:07.463449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:07.463449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:07.463449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:07.464438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:07.464438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:07.464438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:07.464438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:07.464438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:07.464438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:07.464438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:07.464438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:07.464438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:07.464438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:07.464438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:07.464438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:07.464438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:07.464438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:07.464438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:07.464438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:07.464438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:07.464438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:07.465410 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:07.465410 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:07.465410 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:07.465410 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:07.465410 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:07.465410 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:07.465410 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:07.465410 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:07.465410 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 16:23:07.465410 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:07.465410 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 291/582] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:23:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:07.466515 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf47bf9-773a-4795-a95c-fb95d9e9989a None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:07.566596 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:07.572158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80a314-4e53-41a2-b96b-f662539d8faf None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:07.572920 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf47bf9-773a-4795-a95c-fb95d9e9989a None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:07.610842 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0b363-799f-494f-8117-0a8b1dedad58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:07.611469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0b363-799f-494f-8117-0a8b1dedad58 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:07.611750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0b363-799f-494f-8117-0a8b1dedad58 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:07.612017 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0b363-799f-494f-8117-0a8b1dedad58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:07.612382 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6083b9-4c8d-4f35-a915-b402322b30ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:07.612809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0b363-799f-494f-8117-0a8b1dedad58 None admin] RBAC: Adding query filter params (role.id=2aa0cdfed849496db8f3692c9569d7ee, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=bb45ded9f75e4fa2973b564f83bdfaf2) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:07.613065 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6083b9-4c8d-4f35-a915-b402322b30ec None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:07.613164 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0b363-799f-494f-8117-0a8b1dedad58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:07.613342 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6083b9-4c8d-4f35-a915-b402322b30ec None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:07.613643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6083b9-4c8d-4f35-a915-b402322b30ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:07.614589 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0b363-799f-494f-8117-0a8b1dedad58 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:07.620322 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 292/583] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:23:07 2022] GET /identity/v3/role_assignments?user.id=bb45ded9f75e4fa2973b564f83bdfaf2&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=2aa0cdfed849496db8f3692c9569d7ee => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:07.621755 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0b363-799f-494f-8117-0a8b1dedad58 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:07.624568 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:07.630696 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6083b9-4c8d-4f35-a915-b402322b30ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:07.632549 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6083b9-4c8d-4f35-a915-b402322b30ec None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:07.652047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-6d6083b9-4c8d-4f35-a915-b402322b30ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:07.652047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:07.652047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:07.652047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:07.652047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:07.652047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:07.652047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:07.652047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:07.652047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:07.652047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:07.652047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:07.652047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:07.652047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:07.652047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:07.652047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:07.653213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:07.653213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:07.653213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:07.653213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:07.653213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:07.653213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:07.653213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:07.653213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:07.653213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:07.653213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:07.653213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:07.653213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:07.653213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:07.653213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:07.653213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:07.653213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:07.653213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:07.654276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:07.654276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:07.654276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:07.654276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:07.656142 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 292/584] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:07.657308 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6083b9-4c8d-4f35-a915-b402322b30ec None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:07.777978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6083b9-4c8d-4f35-a915-b402322b30ec None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:07.779648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0b363-799f-494f-8117-0a8b1dedad58 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:07.818511 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae76ad-2963-453d-bb0f-5e466f2398d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:07.819421 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae76ad-2963-453d-bb0f-5e466f2398d7 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:07.819925 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae76ad-2963-453d-bb0f-5e466f2398d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:07.820421 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae76ad-2963-453d-bb0f-5e466f2398d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:07.821848 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba76c6c-9278-4d26-afb9-a92228042ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:07.822559 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba76c6c-9278-4d26-afb9-a92228042ced None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:07.823085 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba76c6c-9278-4d26-afb9-a92228042ced None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:07.823445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba76c6c-9278-4d26-afb9-a92228042ced None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:07.825493 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba76c6c-9278-4d26-afb9-a92228042ced None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:07.825879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba76c6c-9278-4d26-afb9-a92228042ced None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:07.827392 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba76c6c-9278-4d26-afb9-a92228042ced None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:07.834657 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae76ad-2963-453d-bb0f-5e466f2398d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:07.836371 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae76ad-2963-453d-bb0f-5e466f2398d7 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:07.849039 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 293/585] 10.222.0.74 () {56 vars in 1137 bytes} [Wed Jul 27 16:23:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:07.850037 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba76c6c-9278-4d26-afb9-a92228042ced None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:07.852766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-2eae76ad-2963-453d-bb0f-5e466f2398d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:07.852766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:07.852766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:07.852766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:07.852766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:07.852766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:07.852766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:07.852766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:07.852766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:07.852766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:07.852766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:07.852766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:07.852766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:07.852766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:07.852766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:07.853658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:07.853658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:07.853658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:07.853658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:07.853658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:07.853658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:07.853658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:07.853658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:07.853658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:07.853658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:07.853658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:07.853658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:07.853658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:07.853658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:07.853658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:07.853658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:07.853658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:07.854517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:07.854517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:07.854517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:07.854517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:07.856671 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 293/586] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:07.857755 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae76ad-2963-453d-bb0f-5e466f2398d7 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:07.864671 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:07.937925 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:07.971557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba76c6c-9278-4d26-afb9-a92228042ced None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:07.997108 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae76ad-2963-453d-bb0f-5e466f2398d7 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:08.013017 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c59ce4-936f-4ca0-8363-ef9ddf966f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:08.013918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c59ce4-936f-4ca0-8363-ef9ddf966f39 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:08.014474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c59ce4-936f-4ca0-8363-ef9ddf966f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:08.014973 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c59ce4-936f-4ca0-8363-ef9ddf966f39 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:08.015946 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c59ce4-936f-4ca0-8363-ef9ddf966f39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:08.016451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c59ce4-936f-4ca0-8363-ef9ddf966f39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:08.018178 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c59ce4-936f-4ca0-8363-ef9ddf966f39 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:08.031419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 294/587] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:08.032474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c59ce4-936f-4ca0-8363-ef9ddf966f39 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:08.050916 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75839e9e-9d61-41fb-ae25-15e40f1a53ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:08.052302 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75839e9e-9d61-41fb-ae25-15e40f1a53ed None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:08.053099 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75839e9e-9d61-41fb-ae25-15e40f1a53ed None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:08.053580 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75839e9e-9d61-41fb-ae25-15e40f1a53ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:08.069056 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75839e9e-9d61-41fb-ae25-15e40f1a53ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:08.071363 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75839e9e-9d61-41fb-ae25-15e40f1a53ed None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:08.090648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-75839e9e-9d61-41fb-ae25-15e40f1a53ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:08.090648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:08.090648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:08.090648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:08.090648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:08.090648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:08.090648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:08.090648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:08.090648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:08.090648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:08.090648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:08.090648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:08.090648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:08.090648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:08.090648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:08.091583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:08.091583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:08.091583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:08.091583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:08.091583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:08.091583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:08.091583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:08.091583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:08.091583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:08.091583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:08.091583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:08.091583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:08.091583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:08.091583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:08.091583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:08.091583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:08.091583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:08.092462 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:08.092462 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:08.092462 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:08.092462 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:08.092462 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 294/588] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:08.093273 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75839e9e-9d61-41fb-ae25-15e40f1a53ed None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:08.123457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:08.165188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c59ce4-936f-4ca0-8363-ef9ddf966f39 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:08.202050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf93d9-a086-407e-9968-a2334f82a140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:08.202680 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf93d9-a086-407e-9968-a2334f82a140 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:08.203007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf93d9-a086-407e-9968-a2334f82a140 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:08.203235 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf93d9-a086-407e-9968-a2334f82a140 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:08.203929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccf93d9-a086-407e-9968-a2334f82a140 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:08.204232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccf93d9-a086-407e-9968-a2334f82a140 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:08.205867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccf93d9-a086-407e-9968-a2334f82a140 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:08.212257 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75839e9e-9d61-41fb-ae25-15e40f1a53ed None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:08.220926 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:08.224147 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 295/589] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:08.225178 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf93d9-a086-407e-9968-a2334f82a140 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:08.254503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b53ec0-cb9a-4eb6-a4f7-b86646c5ded3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:08.255708 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b53ec0-cb9a-4eb6-a4f7-b86646c5ded3 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:08.256292 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b53ec0-cb9a-4eb6-a4f7-b86646c5ded3 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:08.256769 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b53ec0-cb9a-4eb6-a4f7-b86646c5ded3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:08.266096 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b53ec0-cb9a-4eb6-a4f7-b86646c5ded3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:08.267806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b53ec0-cb9a-4eb6-a4f7-b86646c5ded3 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:08.280774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-e2b53ec0-cb9a-4eb6-a4f7-b86646c5ded3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 16:23:08.280774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:08.280774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:08.280774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:08.280774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:08.280774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:08.280774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:08.280774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:08.280774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:08.280774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:08.280774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:08.280774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:08.280774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:08.280774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:08.280774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:08.280774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:08.281750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:08.281750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:08.281750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:08.281750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:08.281750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:08.281750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:08.281750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:08.281750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:08.281750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:08.281750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:08.281750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:08.281750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:08.281750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:08.281750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:08.281750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:08.281750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:08.281750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:08.281750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:08.282678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:08.282678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:08.282678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:08.282678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:08.282678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:08.282678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:08.282678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:08.282678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:08.282678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 16:23:08.282678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:08.285281 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 295/590] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:23:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:08.286364 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b53ec0-cb9a-4eb6-a4f7-b86646c5ded3 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:08.334990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf93d9-a086-407e-9968-a2334f82a140 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:08.363127 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:08.373431 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dcf143-e379-4cf7-bc7c-196c9e6953ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:08.374060 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcf143-e379-4cf7-bc7c-196c9e6953ae None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:08.374345 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcf143-e379-4cf7-bc7c-196c9e6953ae None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:08.374630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcf143-e379-4cf7-bc7c-196c9e6953ae None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:08.375520 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dcf143-e379-4cf7-bc7c-196c9e6953ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:08.375644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dcf143-e379-4cf7-bc7c-196c9e6953ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:08.377046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dcf143-e379-4cf7-bc7c-196c9e6953ae None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:08.390057 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 296/591] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:08.390983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dcf143-e379-4cf7-bc7c-196c9e6953ae None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:08.401392 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b53ec0-cb9a-4eb6-a4f7-b86646c5ded3 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:08.438193 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce083f58-5515-438d-9ff4-f59821f49972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:08.439115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce083f58-5515-438d-9ff4-f59821f49972 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:08.439704 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce083f58-5515-438d-9ff4-f59821f49972 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:08.440185 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce083f58-5515-438d-9ff4-f59821f49972 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:08.448372 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce083f58-5515-438d-9ff4-f59821f49972 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:08.450598 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce083f58-5515-438d-9ff4-f59821f49972 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:08.462729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-ce083f58-5515-438d-9ff4-f59821f49972 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 16:23:08.462729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:08.462729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:08.462729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:08.462729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:08.462729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:08.462729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:08.462729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:08.462729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:08.462729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:08.462729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:08.462729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:08.462729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:08.462729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:08.462729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:08.462729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:08.463625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:08.463625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:08.463625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:08.463625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:08.463625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:08.463625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:08.463625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:08.463625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:08.463625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:08.463625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:08.463625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:08.463625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:08.463625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:08.463625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:08.463625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:08.463625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:08.463625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:08.463625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:08.465124 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:08.465124 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:08.465124 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:08.465124 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:08.465124 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:08.465124 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:08.465124 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:08.465124 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:08.465124 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 16:23:08.465124 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:08.467188 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 296/592] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:23:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:08.468094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce083f58-5515-438d-9ff4-f59821f49972 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:08.496440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:08.502799 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dcf143-e379-4cf7-bc7c-196c9e6953ae None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:08.537540 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b58cf5-dc93-4a5e-b638-11998dc10f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:08.538131 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b58cf5-dc93-4a5e-b638-11998dc10f74 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:08.538601 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b58cf5-dc93-4a5e-b638-11998dc10f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:08.538953 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b58cf5-dc93-4a5e-b638-11998dc10f74 None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:08.539758 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b58cf5-dc93-4a5e-b638-11998dc10f74 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:08.540105 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b58cf5-dc93-4a5e-b638-11998dc10f74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:08.541597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b58cf5-dc93-4a5e-b638-11998dc10f74 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:08.560536 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 297/593] 10.222.0.74 () {56 vars in 1139 bytes} [Wed Jul 27 16:23:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:08.561524 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b58cf5-dc93-4a5e-b638-11998dc10f74 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:08.577047 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce083f58-5515-438d-9ff4-f59821f49972 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:08.612186 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d96a7-2f6b-4c4e-932d-5804cdaedf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:08.613228 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d96a7-2f6b-4c4e-932d-5804cdaedf0f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:08.613737 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d96a7-2f6b-4c4e-932d-5804cdaedf0f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:08.614239 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d96a7-2f6b-4c4e-932d-5804cdaedf0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:08.621985 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d96a7-2f6b-4c4e-932d-5804cdaedf0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:08.623665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d96a7-2f6b-4c4e-932d-5804cdaedf0f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:08.633240 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:08.635718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-987d96a7-2f6b-4c4e-932d-5804cdaedf0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:08.635718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:08.635718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:08.635718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:08.635718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:08.635718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:08.635718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:08.635718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:08.635718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:08.635718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:08.635718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:08.635718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:08.635718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:08.635718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:08.635718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:08.637190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:08.637190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:08.637190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:08.637190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:08.637190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:08.637190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:08.637190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:08.637190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:08.637190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:08.637190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:08.637190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:08.637190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:08.637190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:08.637190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:08.637190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:08.637190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:08.637190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:08.638044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:08.638044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:08.638044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:08.638044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:08.638044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:08.638044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:08.638044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:08.638044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:08.640381 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 297/594] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:08.641490 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d96a7-2f6b-4c4e-932d-5804cdaedf0f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:08.669161 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b58cf5-dc93-4a5e-b638-11998dc10f74 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:08.705671 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28c99d-11e9-4e08-abc6-934fba211fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:08.706088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28c99d-11e9-4e08-abc6-934fba211fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:08.706378 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28c99d-11e9-4e08-abc6-934fba211fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:08.706657 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28c99d-11e9-4e08-abc6-934fba211fe8 None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:08.707450 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28c99d-11e9-4e08-abc6-934fba211fe8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:08.707821 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28c99d-11e9-4e08-abc6-934fba211fe8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:08.709214 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28c99d-11e9-4e08-abc6-934fba211fe8 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:08.732199 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 298/595] 10.222.0.74 () {56 vars in 1137 bytes} [Wed Jul 27 16:23:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:08.733043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28c99d-11e9-4e08-abc6-934fba211fe8 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:08.736403 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:08.753697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d96a7-2f6b-4c4e-932d-5804cdaedf0f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:08.791310 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3cf579-ca00-4510-bb9c-17baa5bd05f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:08.792341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cf579-ca00-4510-bb9c-17baa5bd05f7 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:08.793386 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cf579-ca00-4510-bb9c-17baa5bd05f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:08.793878 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cf579-ca00-4510-bb9c-17baa5bd05f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:08.805241 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3cf579-ca00-4510-bb9c-17baa5bd05f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:08.806975 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3cf579-ca00-4510-bb9c-17baa5bd05f7 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:08.821955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-fe3cf579-ca00-4510-bb9c-17baa5bd05f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:08.821955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:08.821955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:08.821955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:08.821955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:08.821955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:08.821955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:08.821955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:08.821955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:08.821955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:08.821955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:08.821955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:08.821955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:08.821955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:08.821955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:08.822858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:08.822858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:08.822858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:08.822858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:08.822858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:08.822858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:08.822858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:08.822858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:08.822858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:08.822858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:08.822858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:08.822858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:08.822858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:08.822858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:08.822858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:08.822858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:08.822858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:08.824260 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:08.824260 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:08.824260 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:08.824260 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:08.826373 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 298/596] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:08.827352 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3cf579-ca00-4510-bb9c-17baa5bd05f7 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:08.840517 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28c99d-11e9-4e08-abc6-934fba211fe8 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:08.857320 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:08.878810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c6a5e-2f9e-4538-9d5b-abc7b00df406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:08.879157 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c6a5e-2f9e-4538-9d5b-abc7b00df406 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:08.879446 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c6a5e-2f9e-4538-9d5b-abc7b00df406 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:08.879727 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c6a5e-2f9e-4538-9d5b-abc7b00df406 None admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:08.880485 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c6a5e-2f9e-4538-9d5b-abc7b00df406 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:08.880798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c6a5e-2f9e-4538-9d5b-abc7b00df406 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:08.882285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c6a5e-2f9e-4538-9d5b-abc7b00df406 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:08.896707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 299/597] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:08.898168 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c6a5e-2f9e-4538-9d5b-abc7b00df406 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:08.941947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3cf579-ca00-4510-bb9c-17baa5bd05f7 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:08.978016 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbe8b6-3387-4085-9602-f04335d6c3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:08.979173 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbe8b6-3387-4085-9602-f04335d6c3ed None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:08.979718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbe8b6-3387-4085-9602-f04335d6c3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:08.980230 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbe8b6-3387-4085-9602-f04335d6c3ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:08.987785 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:08.992641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbbe8b6-3387-4085-9602-f04335d6c3ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:08.995261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbbe8b6-3387-4085-9602-f04335d6c3ed None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:09.003848 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c6a5e-2f9e-4538-9d5b-abc7b00df406 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:09.014979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-1bbbe8b6-3387-4085-9602-f04335d6c3ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:09.014979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:09.014979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:09.014979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:09.014979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:09.014979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:09.014979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:09.014979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:09.014979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:09.014979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:09.014979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:09.014979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:09.014979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:09.014979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:09.014979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:09.015856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:09.015856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:09.015856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:09.015856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:09.015856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:09.015856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:09.015856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:09.015856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:09.015856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:09.015856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:09.015856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:09.015856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:09.015856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:09.015856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:09.015856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:09.015856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:09.015856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:09.016728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:09.016728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:09.016728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:09.016728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:09.019051 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 299/598] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:09.020004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbe8b6-3387-4085-9602-f04335d6c3ed None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:09.045521 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b9b22-a66f-4476-aec4-ea9e3584af05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:09.046148 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b9b22-a66f-4476-aec4-ea9e3584af05 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:09.046434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b9b22-a66f-4476-aec4-ea9e3584af05 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:09.046713 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b9b22-a66f-4476-aec4-ea9e3584af05 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:09.047406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b9b22-a66f-4476-aec4-ea9e3584af05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:09.047714 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b9b22-a66f-4476-aec4-ea9e3584af05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:09.049314 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b9b22-a66f-4476-aec4-ea9e3584af05 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:09.062188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 300/599] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:09.063161 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b9b22-a66f-4476-aec4-ea9e3584af05 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:09.087832 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:09.135571 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbe8b6-3387-4085-9602-f04335d6c3ed None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:09.170668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b9b22-a66f-4476-aec4-ea9e3584af05 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:09.176055 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e45f7f-674a-4216-a2fb-9e341ef3e519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:09.176950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e45f7f-674a-4216-a2fb-9e341ef3e519 None admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:09.177433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e45f7f-674a-4216-a2fb-9e341ef3e519 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:09.178002 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e45f7f-674a-4216-a2fb-9e341ef3e519 None admin] PATH_INFO: `/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/c50c2d1781b04a5fa8be81bce9205747/roles/2aa0cdfed849496db8f3692c9569d7ee` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:09.194103 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e45f7f-674a-4216-a2fb-9e341ef3e519 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d6bcdd09cf24a5c8003951db37020cf, user_id=c50c2d1781b04a5fa8be81bce9205747, role_id=2aa0cdfed849496db8f3692c9569d7ee)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:09.195813 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e45f7f-674a-4216-a2fb-9e341ef3e519 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:09.211012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 300/600] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:23:09 2022] PUT /identity/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/c50c2d1781b04a5fa8be81bce9205747/roles/2aa0cdfed849496db8f3692c9569d7ee => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:23:09.211655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372725e5-c654-4716-907e-b0fe227702ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:09.212272 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e45f7f-674a-4216-a2fb-9e341ef3e519 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:09.213028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372725e5-c654-4716-907e-b0fe227702ec None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:09.213652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372725e5-c654-4716-907e-b0fe227702ec None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:09.213891 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372725e5-c654-4716-907e-b0fe227702ec None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:09.215286 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372725e5-c654-4716-907e-b0fe227702ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:09.215662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372725e5-c654-4716-907e-b0fe227702ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:09.217383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372725e5-c654-4716-907e-b0fe227702ec None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:09.231392 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 301/601] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:09.232851 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485071a5-3030-4d38-a731-5ac8aa3e86a3 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:09.233264 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485071a5-3030-4d38-a731-5ac8aa3e86a3 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:09.233630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485071a5-3030-4d38-a731-5ac8aa3e86a3 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:09.239725 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 302/602] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:09.240592 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485071a5-3030-4d38-a731-5ac8aa3e86a3 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:09.268185 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:09.343008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e45f7f-674a-4216-a2fb-9e341ef3e519 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:09.362546 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:09.390610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485071a5-3030-4d38-a731-5ac8aa3e86a3 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:09.391454 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff73e30-0927-4d69-8f54-1f3824a831df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:09.392171 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff73e30-0927-4d69-8f54-1f3824a831df None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:09.392459 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff73e30-0927-4d69-8f54-1f3824a831df None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:09.392736 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff73e30-0927-4d69-8f54-1f3824a831df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:09.401824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff73e30-0927-4d69-8f54-1f3824a831df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:09.403710 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff73e30-0927-4d69-8f54-1f3824a831df None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:09.419429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-6ff73e30-0927-4d69-8f54-1f3824a831df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:09.419429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:09.419429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:09.419429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:09.419429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:09.419429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:09.419429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:09.419429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:09.419429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:09.419429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:09.419429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:09.419429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:09.419429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:09.419429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:09.419429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:09.420378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:09.420378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:09.420378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:09.420378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:09.420378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:09.420378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:09.420378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:09.420378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:09.420378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:09.420378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:09.420378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:09.420378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:09.420378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:09.420378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:09.420378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:09.420378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:09.420378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:09.421669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:09.421669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:09.421669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:09.421669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:09.421669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:09.421669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:09.421669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:09.421669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:09.421669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 301/603] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:09.423946 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac38573-c170-49c2-9053-fe24ed92e081 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:09.423946 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac38573-c170-49c2-9053-fe24ed92e081 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:09.423946 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac38573-c170-49c2-9053-fe24ed92e081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:09.434124 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933cc903-bba9-41dc-add5-98e9fca32528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:09.435109 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cc903-bba9-41dc-add5-98e9fca32528 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:09.435395 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cc903-bba9-41dc-add5-98e9fca32528 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:09.435669 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cc903-bba9-41dc-add5-98e9fca32528 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:09.444217 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-9ac38573-c170-49c2-9053-fe24ed92e081 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:09.444525 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933cc903-bba9-41dc-add5-98e9fca32528 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:09.445451 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9ac38573-c170-49c2-9053-fe24ed92e081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:09.446031 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933cc903-bba9-41dc-add5-98e9fca32528 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:09.459651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-933cc903-bba9-41dc-add5-98e9fca32528 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:09.459651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:09.459651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:09.459651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:09.459651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:09.459651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:09.459651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:09.459651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:09.459651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:09.459651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:09.459651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:09.459651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:09.459651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:09.459651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:09.459651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:09.460579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:09.460579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:09.460579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:09.460579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:09.460579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:09.460579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:09.460579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:09.460579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:09.460579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:09.460579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:09.460579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:09.460579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:09.460579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:09.460579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:09.460579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:09.460579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:09.460579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:09.461506 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:09.461506 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:09.461506 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:09.461506 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:09.461506 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:09.461506 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:09.461506 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:09.461506 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:09.461506 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 303/604] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:23:09.461995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933cc903-bba9-41dc-add5-98e9fca32528 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:09.537410 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 302/605] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:23:09.538275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac38573-c170-49c2-9053-fe24ed92e081 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:09.573403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933cc903-bba9-41dc-add5-98e9fca32528 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:09.582952 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:09.611432 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc078590-6151-406b-9302-3de12c0fa29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:09.612223 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc078590-6151-406b-9302-3de12c0fa29d None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:09.612608 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc078590-6151-406b-9302-3de12c0fa29d None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:09.613032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc078590-6151-406b-9302-3de12c0fa29d None admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:09.613892 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc078590-6151-406b-9302-3de12c0fa29d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:09.614303 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc078590-6151-406b-9302-3de12c0fa29d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:09.615840 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc078590-6151-406b-9302-3de12c0fa29d None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:09.629347 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 304/606] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:09.630419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc078590-6151-406b-9302-3de12c0fa29d None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:09.633938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:09.674768 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac38573-c170-49c2-9053-fe24ed92e081 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:09.719854 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485483c0-e99c-4069-83dc-43012a91890a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:09.720476 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485483c0-e99c-4069-83dc-43012a91890a None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:09.720864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485483c0-e99c-4069-83dc-43012a91890a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:09.721070 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485483c0-e99c-4069-83dc-43012a91890a None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:09.722107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485483c0-e99c-4069-83dc-43012a91890a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:09.722107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485483c0-e99c-4069-83dc-43012a91890a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:09.723534 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485483c0-e99c-4069-83dc-43012a91890a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:09.736048 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 303/607] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:09.737404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485483c0-e99c-4069-83dc-43012a91890a None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:09.741373 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc078590-6151-406b-9302-3de12c0fa29d None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:09.779712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5124348-5e64-4dd1-a4cd-12e9cd22beaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:09.780405 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5124348-5e64-4dd1-a4cd-12e9cd22beaa None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:09.780692 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5124348-5e64-4dd1-a4cd-12e9cd22beaa None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:09.780994 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5124348-5e64-4dd1-a4cd-12e9cd22beaa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:09.792321 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5124348-5e64-4dd1-a4cd-12e9cd22beaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:09.797830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-c5124348-5e64-4dd1-a4cd-12e9cd22beaa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:09.797830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:09.797830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:09.797830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:09.797830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:09.797830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:09.797830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:09.797830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:09.797830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:09.797830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:09.797830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:09.797830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:09.797830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:09.797830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:09.797830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:09.797830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:09.798796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:09.798796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:09.798796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:09.798796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:09.798796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:09.798796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:09.798796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:09.798796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:09.798796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:09.798796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:09.798796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:09.798796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:09.798796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:09.798796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:09.798796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:09.798796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:09.799646 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:09.799646 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:09.799646 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:09.799646 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:09.799646 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:09.799646 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:09.799646 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:09.799646 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:09.799646 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:09.799646 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:09.799646 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:09.799646 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:09.799646 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 305/608] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:23:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:09.801240 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb22cc9-b276-4605-88fb-3ce6e9880ad9 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:09.801240 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb22cc9-b276-4605-88fb-3ce6e9880ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:09.801240 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb22cc9-b276-4605-88fb-3ce6e9880ad9 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:09.805951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 306/609] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:09.806554 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb22cc9-b276-4605-88fb-3ce6e9880ad9 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:09.846984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:09.858449 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485483c0-e99c-4069-83dc-43012a91890a None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:09.875553 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:09.900646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7eb80-0d79-4ae6-826e-5a2adf6c92dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:09.901343 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7eb80-0d79-4ae6-826e-5a2adf6c92dd None admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:09.901665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7eb80-0d79-4ae6-826e-5a2adf6c92dd None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:09.902007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7eb80-0d79-4ae6-826e-5a2adf6c92dd None admin] PATH_INFO: `/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/193dd9fee1844275ad0dab602e93df11/roles/2aa0cdfed849496db8f3692c9569d7ee` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:09.916919 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7eb80-0d79-4ae6-826e-5a2adf6c92dd None admin] RBAC: Authorizing `identity:create_grant(project_id=8d6bcdd09cf24a5c8003951db37020cf, user_id=193dd9fee1844275ad0dab602e93df11, role_id=2aa0cdfed849496db8f3692c9569d7ee)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:09.918409 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7eb80-0d79-4ae6-826e-5a2adf6c92dd None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:09.919306 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb22cc9-b276-4605-88fb-3ce6e9880ad9 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:09.940927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 304/610] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:23:09 2022] PUT /identity/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/193dd9fee1844275ad0dab602e93df11/roles/2aa0cdfed849496db8f3692c9569d7ee => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:23:09.940927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7eb80-0d79-4ae6-826e-5a2adf6c92dd None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:09.972589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4905d9a-2b00-4c5a-82f7-e6de22b331b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:09.974265 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4905d9a-2b00-4c5a-82f7-e6de22b331b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:09.974604 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4905d9a-2b00-4c5a-82f7-e6de22b331b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:09.974887 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4905d9a-2b00-4c5a-82f7-e6de22b331b1 None admin] PATH_INFO: `/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/dd352d423bff418f932b434c51b15d23/roles/2aa0cdfed849496db8f3692c9569d7ee` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:09.995663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4905d9a-2b00-4c5a-82f7-e6de22b331b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d6bcdd09cf24a5c8003951db37020cf, user_id=dd352d423bff418f932b434c51b15d23, role_id=2aa0cdfed849496db8f3692c9569d7ee)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:09.997855 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4905d9a-2b00-4c5a-82f7-e6de22b331b1 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:10.020982 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 307/611] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:23:09 2022] PUT /identity/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/dd352d423bff418f932b434c51b15d23/roles/2aa0cdfed849496db8f3692c9569d7ee => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:23:10.022636 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639cae4f-536a-4fb4-b6fb-6c1b3a8d4817 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:10.022924 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639cae4f-536a-4fb4-b6fb-6c1b3a8d4817 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:10.023200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639cae4f-536a-4fb4-b6fb-6c1b3a8d4817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:10.062108 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-639cae4f-536a-4fb4-b6fb-6c1b3a8d4817 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:10.063254 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-639cae4f-536a-4fb4-b6fb-6c1b3a8d4817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:10.082256 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7eb80-0d79-4ae6-826e-5a2adf6c92dd None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:10.148752 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffa942-fc2b-4959-b262-180c9b48fa3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:10.149430 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffa942-fc2b-4959-b262-180c9b48fa3d None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:10.149714 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffa942-fc2b-4959-b262-180c9b48fa3d None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:10.149989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffa942-fc2b-4959-b262-180c9b48fa3d None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:10.150995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ffa942-fc2b-4959-b262-180c9b48fa3d None admin] RBAC: Adding query filter params (name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:10.150995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ffa942-fc2b-4959-b262-180c9b48fa3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:10.152526 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ffa942-fc2b-4959-b262-180c9b48fa3d None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:10.162803 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 305/612] 10.222.0.74 () {56 vars in 1103 bytes} [Wed Jul 27 16:23:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:10.170382 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffa942-fc2b-4959-b262-180c9b48fa3d None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:10.173665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:10.191537 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 308/613] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:23:10.226592 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639cae4f-536a-4fb4-b6fb-6c1b3a8d4817 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:10.280321 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:10.349102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639cae4f-536a-4fb4-b6fb-6c1b3a8d4817 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:10.367265 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffa942-fc2b-4959-b262-180c9b48fa3d None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:10.394459 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928084fe-b316-49a1-83a4-7664172b5176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:10.395434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928084fe-b316-49a1-83a4-7664172b5176 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:10.396337 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928084fe-b316-49a1-83a4-7664172b5176 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:10.396799 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928084fe-b316-49a1-83a4-7664172b5176 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:10.410094 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928084fe-b316-49a1-83a4-7664172b5176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:10.415910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-928084fe-b316-49a1-83a4-7664172b5176 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:10.415910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:10.415910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:10.415910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:10.415910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:10.415910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:10.415910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:10.415910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:10.415910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:10.415910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:10.415910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:10.415910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:10.415910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:10.415910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:10.415910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:10.415910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:10.416822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:10.416822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:10.416822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:10.416822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:10.416822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:10.416822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:10.416822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:10.416822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:10.416822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:10.416822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:10.416822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:10.416822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:10.416822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:10.416822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:10.416822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:10.416822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:10.417748 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:10.417748 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:10.417748 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:10.417748 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:10.417748 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:10.417748 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:10.417748 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:10.417748 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:10.417748 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:10.417748 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:10.417748 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:10.417748 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:10.421677 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 309/614] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:23:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:23:10.434099 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928084fe-b316-49a1-83a4-7664172b5176 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:10.440492 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990aa7b-1441-4043-8903-8622c759b12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:10.441180 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990aa7b-1441-4043-8903-8622c759b12f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:10.441465 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990aa7b-1441-4043-8903-8622c759b12f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:10.441748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990aa7b-1441-4043-8903-8622c759b12f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:10.456423 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6990aa7b-1441-4043-8903-8622c759b12f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:10.457915 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6990aa7b-1441-4043-8903-8622c759b12f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:10.484264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-6990aa7b-1441-4043-8903-8622c759b12f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:10.484264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:10.484264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:10.484264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:10.484264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:10.484264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:10.484264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:10.484264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:10.484264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:10.484264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:10.484264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:10.484264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:10.484264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:10.484264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:10.484264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:10.485222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:10.485222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:10.485222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:10.485222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:10.485222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:10.485222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:10.485222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:10.485222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:10.485222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:10.485222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:10.485222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:10.485222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:10.485222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:10.485222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:10.485222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:10.485222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:10.485222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:10.486099 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:10.486099 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:10.486099 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:10.486099 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:10.486099 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 306/615] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:10.493829 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990aa7b-1441-4043-8903-8622c759b12f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:10.581937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928084fe-b316-49a1-83a4-7664172b5176 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:10.583708 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:10.610599 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:10.630900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e86e9d-c066-43b7-ac71-e6c0e07dea9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:10.631533 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e86e9d-c066-43b7-ac71-e6c0e07dea9a None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:10.631817 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e86e9d-c066-43b7-ac71-e6c0e07dea9a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:10.632099 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e86e9d-c066-43b7-ac71-e6c0e07dea9a None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:10.632827 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e86e9d-c066-43b7-ac71-e6c0e07dea9a None admin] RBAC: Adding query filter params (name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:10.633163 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e86e9d-c066-43b7-ac71-e6c0e07dea9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:10.634543 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e86e9d-c066-43b7-ac71-e6c0e07dea9a None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:10.645244 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 310/616] 10.222.0.74 () {56 vars in 1103 bytes} [Wed Jul 27 16:23:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:10.650524 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e86e9d-c066-43b7-ac71-e6c0e07dea9a None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:10.684821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990aa7b-1441-4043-8903-8622c759b12f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:10.722005 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fde3e0-3cb0-4f93-a879-1155f135ac39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:10.722627 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fde3e0-3cb0-4f93-a879-1155f135ac39 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:10.722909 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fde3e0-3cb0-4f93-a879-1155f135ac39 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:10.723293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fde3e0-3cb0-4f93-a879-1155f135ac39 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:10.723987 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fde3e0-3cb0-4f93-a879-1155f135ac39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:10.724294 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fde3e0-3cb0-4f93-a879-1155f135ac39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:10.725814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fde3e0-3cb0-4f93-a879-1155f135ac39 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:10.738948 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 307/617] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:10.744160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fde3e0-3cb0-4f93-a879-1155f135ac39 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:10.826514 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e86e9d-c066-43b7-ac71-e6c0e07dea9a None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:10.891549 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f1e2d-4bed-482b-8698-7a2400734efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:10.892122 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f1e2d-4bed-482b-8698-7a2400734efe None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:10.892407 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f1e2d-4bed-482b-8698-7a2400734efe None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:10.892688 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f1e2d-4bed-482b-8698-7a2400734efe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:10.907768 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f1e2d-4bed-482b-8698-7a2400734efe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:10.909285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f1e2d-4bed-482b-8698-7a2400734efe None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:10.938451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-878f1e2d-4bed-482b-8698-7a2400734efe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:10.938451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:10.938451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:10.938451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:10.938451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:10.938451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:10.938451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:10.938451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:10.938451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:10.938451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:10.938451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:10.938451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:10.938451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:10.938451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:10.938451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:10.939375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:10.939375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:10.939375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:10.939375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:10.939375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:10.939375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:10.939375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:10.939375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:10.939375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:10.939375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:10.939375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:10.939375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:10.939375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:10.939375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:10.939375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:10.939375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:10.939375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:10.940339 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:10.940339 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:10.940339 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:10.940339 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:10.946745 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 311/618] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:10.951712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f1e2d-4bed-482b-8698-7a2400734efe None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:10.964446 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fde3e0-3cb0-4f93-a879-1155f135ac39 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:10.984883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:10.992759 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:11.020295 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26f01b-4cc7-41bc-a627-6fb6cde39058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:11.021442 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26f01b-4cc7-41bc-a627-6fb6cde39058 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:11.021933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26f01b-4cc7-41bc-a627-6fb6cde39058 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:11.022411 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26f01b-4cc7-41bc-a627-6fb6cde39058 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:11.032116 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26f01b-4cc7-41bc-a627-6fb6cde39058 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:11.033837 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26f01b-4cc7-41bc-a627-6fb6cde39058 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:11.047149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-6a26f01b-4cc7-41bc-a627-6fb6cde39058 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 16:23:11.047149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:11.047149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:11.047149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:11.047149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:11.047149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:11.047149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:11.047149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:11.047149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:11.047149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:11.047149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:11.047149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:11.047149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:11.047149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:11.047149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:11.047149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:11.048065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:11.048065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:11.048065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:11.048065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:11.048065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:11.048065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:11.048065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:11.048065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:11.048065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:11.048065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:11.048065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:11.048065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:11.048065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:11.048065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:11.048065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:11.048065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:11.048065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:11.048065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:11.048985 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:11.048985 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:11.048985 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:11.048985 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:11.048985 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:11.048985 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:11.048985 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:11.048985 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:11.048985 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 16:23:11.048985 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:11.051900 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 308/619] 10.222.0.74 () {56 vars in 1093 bytes} [Wed Jul 27 16:23:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:11.058301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26f01b-4cc7-41bc-a627-6fb6cde39058 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:11.178981 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26f01b-4cc7-41bc-a627-6fb6cde39058 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:11.183515 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f1e2d-4bed-482b-8698-7a2400734efe None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:11.220607 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c1714-3a21-442d-9e9d-5e9971dc7551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:11.221674 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c1714-3a21-442d-9e9d-5e9971dc7551 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:11.222491 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c1714-3a21-442d-9e9d-5e9971dc7551 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:11.222970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c1714-3a21-442d-9e9d-5e9971dc7551 None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:11.223977 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c1714-3a21-442d-9e9d-5e9971dc7551 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:11.224490 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c1714-3a21-442d-9e9d-5e9971dc7551 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:11.226229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c1714-3a21-442d-9e9d-5e9971dc7551 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:11.247842 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 309/620] 10.222.0.74 () {56 vars in 1133 bytes} [Wed Jul 27 16:23:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:11.253769 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c1714-3a21-442d-9e9d-5e9971dc7551 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:11.276472 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4343e76-ae46-4b21-a8bd-0560e66f0e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:11.277217 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4343e76-ae46-4b21-a8bd-0560e66f0e0d None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:11.277483 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4343e76-ae46-4b21-a8bd-0560e66f0e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:11.277753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4343e76-ae46-4b21-a8bd-0560e66f0e0d None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:11.278463 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4343e76-ae46-4b21-a8bd-0560e66f0e0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:11.278766 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4343e76-ae46-4b21-a8bd-0560e66f0e0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:11.280196 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4343e76-ae46-4b21-a8bd-0560e66f0e0d None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:11.294418 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:11.321219 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 312/621] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:11.326895 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4343e76-ae46-4b21-a8bd-0560e66f0e0d None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:11.432034 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c1714-3a21-442d-9e9d-5e9971dc7551 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:11.471145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41aadf6-e752-4db8-aa36-ce9916e87aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:11.471771 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41aadf6-e752-4db8-aa36-ce9916e87aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:11.472054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41aadf6-e752-4db8-aa36-ce9916e87aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:11.472329 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41aadf6-e752-4db8-aa36-ce9916e87aa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:11.487253 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:11.533595 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4343e76-ae46-4b21-a8bd-0560e66f0e0d None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:11.546231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41aadf6-e752-4db8-aa36-ce9916e87aa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:11.547803 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41aadf6-e752-4db8-aa36-ce9916e87aa6 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:11.581366 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:11.595158 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5740e-c6fb-40da-b079-eed31932dd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:11.596032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5740e-c6fb-40da-b079-eed31932dd26 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:11.596463 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5740e-c6fb-40da-b079-eed31932dd26 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:11.602998 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5740e-c6fb-40da-b079-eed31932dd26 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:11.610772 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd5740e-c6fb-40da-b079-eed31932dd26 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:11.621007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd5740e-c6fb-40da-b079-eed31932dd26 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:11.641662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-fdd5740e-c6fb-40da-b079-eed31932dd26 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 16:23:11.641662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:11.641662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:11.641662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:11.641662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:11.641662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:11.641662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:11.641662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:11.641662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:11.641662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:11.641662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:11.641662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:11.641662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:11.641662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:11.641662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:11.641662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:11.642853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:11.642853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:11.642853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:11.642853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:11.642853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:11.642853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:11.642853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:11.642853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:11.642853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:11.642853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:11.642853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:11.642853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:11.642853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:11.642853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:11.642853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:11.642853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:11.642853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:11.642853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:11.644007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:11.644007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:11.644007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:11.644007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:11.644007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:11.644007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:11.644007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:11.644007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:11.644007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 16:23:11.644007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:11.646145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 313/622] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:23:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:23:11.651191 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5740e-c6fb-40da-b079-eed31932dd26 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:11.663028 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 310/623] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:23:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:23:11.665968 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41aadf6-e752-4db8-aa36-ce9916e87aa6 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:11.761029 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5740e-c6fb-40da-b079-eed31932dd26 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:11.773892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41aadf6-e752-4db8-aa36-ce9916e87aa6 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:11.797109 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d116f7-967e-4039-b438-a11a70dd46fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:11.798090 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d116f7-967e-4039-b438-a11a70dd46fa None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:11.798597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d116f7-967e-4039-b438-a11a70dd46fa None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:11.799075 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d116f7-967e-4039-b438-a11a70dd46fa None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:11.800090 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d116f7-967e-4039-b438-a11a70dd46fa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:11.800601 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d116f7-967e-4039-b438-a11a70dd46fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:11.802259 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d116f7-967e-4039-b438-a11a70dd46fa None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:11.805695 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:11.806643 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:11.815604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208c511-dc8d-4f11-9d27-7eda2a7036cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:11.815816 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208c511-dc8d-4f11-9d27-7eda2a7036cc None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:11.816031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208c511-dc8d-4f11-9d27-7eda2a7036cc None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:11.816268 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208c511-dc8d-4f11-9d27-7eda2a7036cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:11.824932 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 314/624] 10.222.0.74 () {56 vars in 1135 bytes} [Wed Jul 27 16:23:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:11.826714 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb576c-684d-4311-b11b-5580972fad6c None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:11.827226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb576c-684d-4311-b11b-5580972fad6c None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:11.827578 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208c511-dc8d-4f11-9d27-7eda2a7036cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:11.827925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb576c-684d-4311-b11b-5580972fad6c None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:11.829118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208c511-dc8d-4f11-9d27-7eda2a7036cc None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:11.830030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 315/625] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:11.830945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb576c-684d-4311-b11b-5580972fad6c None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:11.849404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-9208c511-dc8d-4f11-9d27-7eda2a7036cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:11.849404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:11.849404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:11.849404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:11.849404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:11.849404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:11.849404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:11.849404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:11.849404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:11.849404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:11.849404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:11.849404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:11.849404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:11.849404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:11.849404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:11.850377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:11.850377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:11.850377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:11.850377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:11.850377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:11.850377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:11.850377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:11.850377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:11.850377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:11.850377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:11.850377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:11.850377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:11.850377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:11.850377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:11.850377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:11.850377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:11.850377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:11.851261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:11.851261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:11.851261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:11.851261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:11.851261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 311/626] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:11.856549 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a163b-304b-4988-bfb8-f110ae590bb0 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:11.856549 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a163b-304b-4988-bfb8-f110ae590bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:11.856549 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a163b-304b-4988-bfb8-f110ae590bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:11.873489 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-bb9a163b-304b-4988-bfb8-f110ae590bb0 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:11.874859 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-bb9a163b-304b-4988-bfb8-f110ae590bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:11.946234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb576c-684d-4311-b11b-5580972fad6c None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:11.948505 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 312/627] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:23:11.957927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a163b-304b-4988-bfb8-f110ae590bb0 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:11.983618 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa352c48-2bab-4058-ac36-964fc41cdcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:11.984508 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa352c48-2bab-4058-ac36-964fc41cdcac None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:11.985033 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa352c48-2bab-4058-ac36-964fc41cdcac None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:11.985498 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa352c48-2bab-4058-ac36-964fc41cdcac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:11.996308 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa352c48-2bab-4058-ac36-964fc41cdcac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:11.998075 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa352c48-2bab-4058-ac36-964fc41cdcac None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:12.014536 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-fa352c48-2bab-4058-ac36-964fc41cdcac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:12.014536 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:12.014536 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:12.014536 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:12.014536 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:12.014536 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:12.014536 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:12.014536 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:12.014536 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:12.014536 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:12.014536 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:12.014536 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:12.014536 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:12.014536 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:12.014536 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:12.015532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:12.015532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:12.015532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:12.015532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:12.015532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:12.015532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:12.015532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:12.015532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:12.015532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:12.015532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:12.015532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:12.015532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:12.015532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:12.015532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:12.015532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:12.015532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:12.015532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:12.016398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:12.016398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:12.016398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:12.016398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:12.018691 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 316/628] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:12.019658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa352c48-2bab-4058-ac36-964fc41cdcac None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:12.069123 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:12.084551 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a163b-304b-4988-bfb8-f110ae590bb0 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:12.100312 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:12.124441 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d78c3-abaf-44a8-93a6-a98fad3c80c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:12.125093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d78c3-abaf-44a8-93a6-a98fad3c80c3 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:12.125372 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d78c3-abaf-44a8-93a6-a98fad3c80c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:12.125648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d78c3-abaf-44a8-93a6-a98fad3c80c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:12.126334 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d78c3-abaf-44a8-93a6-a98fad3c80c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:12.126639 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d78c3-abaf-44a8-93a6-a98fad3c80c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:12.128012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d78c3-abaf-44a8-93a6-a98fad3c80c3 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:12.128765 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa352c48-2bab-4058-ac36-964fc41cdcac None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:12.144630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 313/629] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:12.144630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d78c3-abaf-44a8-93a6-a98fad3c80c3 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:12.169913 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b20f4-c620-4949-8d94-bdded2227324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:12.171044 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b20f4-c620-4949-8d94-bdded2227324 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:12.171475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b20f4-c620-4949-8d94-bdded2227324 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:12.171922 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b20f4-c620-4949-8d94-bdded2227324 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:12.182706 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329b20f4-c620-4949-8d94-bdded2227324 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:12.188412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-329b20f4-c620-4949-8d94-bdded2227324 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:12.188412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:12.188412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:12.188412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:12.188412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:12.188412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:12.188412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:12.188412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:12.188412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:12.188412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:12.188412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:12.188412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:12.188412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:12.188412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:12.188412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:12.188412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:12.189355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:12.189355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:12.189355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:12.189355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:12.189355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:12.189355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:12.189355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:12.189355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:12.189355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:12.189355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:12.189355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:12.189355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:12.189355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:12.189355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:12.189355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:12.189355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:12.190192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:12.190192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:12.190192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:12.190192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:12.190192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:12.190192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:12.190192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:12.190192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:12.190192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:12.190192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:12.190192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:12.190192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:12.192851 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 317/630] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:23:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:12.193794 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b20f4-c620-4949-8d94-bdded2227324 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:12.258359 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d78c3-abaf-44a8-93a6-a98fad3c80c3 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:12.295888 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda66438-fc2a-42d2-871c-fc03587202a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:12.296431 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda66438-fc2a-42d2-871c-fc03587202a6 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:12.296709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda66438-fc2a-42d2-871c-fc03587202a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:12.297004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda66438-fc2a-42d2-871c-fc03587202a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:12.297701 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda66438-fc2a-42d2-871c-fc03587202a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:12.298004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda66438-fc2a-42d2-871c-fc03587202a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:12.299371 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda66438-fc2a-42d2-871c-fc03587202a6 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:12.304437 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b20f4-c620-4949-8d94-bdded2227324 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:12.311998 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:12.316576 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 314/631] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:12.316576 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda66438-fc2a-42d2-871c-fc03587202a6 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:12.346648 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f9f2a-ee00-44ed-b3ef-5b7b857c286b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:12.347538 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f9f2a-ee00-44ed-b3ef-5b7b857c286b None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:12.347972 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f9f2a-ee00-44ed-b3ef-5b7b857c286b None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:12.348423 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f9f2a-ee00-44ed-b3ef-5b7b857c286b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:12.349246 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:12.357804 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f9f2a-ee00-44ed-b3ef-5b7b857c286b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:12.359265 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f9f2a-ee00-44ed-b3ef-5b7b857c286b None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:12.370397 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-7b8f9f2a-ee00-44ed-b3ef-5b7b857c286b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:12.370397 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:12.370397 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:12.370397 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:12.370397 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:12.370397 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:12.370397 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:12.370397 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:12.370397 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:12.370397 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:12.370397 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:12.370397 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:12.370397 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:12.370397 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:12.370397 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:12.371833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:12.371833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:12.371833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:12.371833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:12.371833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:12.371833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:12.371833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:12.371833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:12.371833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:12.371833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:12.371833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:12.371833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:12.371833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:12.371833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:12.371833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:12.371833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:12.371833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:12.372911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:12.372911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:12.372911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:12.372911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:12.372911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:12.372911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:12.372911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:12.372911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:12.372911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 318/632] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:12.373761 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f950630-f662-4e6e-a14e-8dfc5299601f None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:12.373761 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f950630-f662-4e6e-a14e-8dfc5299601f None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:12.373761 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f950630-f662-4e6e-a14e-8dfc5299601f None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:12.374963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 319/633] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:12.375713 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f950630-f662-4e6e-a14e-8dfc5299601f None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:12.421228 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda66438-fc2a-42d2-871c-fc03587202a6 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:12.455043 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926550db-c5b6-42ae-a999-5b14824aa939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:12.455652 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926550db-c5b6-42ae-a999-5b14824aa939 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:12.455920 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926550db-c5b6-42ae-a999-5b14824aa939 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:12.456189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926550db-c5b6-42ae-a999-5b14824aa939 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:12.456915 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926550db-c5b6-42ae-a999-5b14824aa939 None admin] RBAC: Adding query filter params (name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:12.457215 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926550db-c5b6-42ae-a999-5b14824aa939 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:12.458569 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926550db-c5b6-42ae-a999-5b14824aa939 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:12.466932 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 315/634] 10.222.0.74 () {56 vars in 1103 bytes} [Wed Jul 27 16:23:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:12.467924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a6b1b-7a1e-4e9d-9d04-595566b6e9ff None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:12.468208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a6b1b-7a1e-4e9d-9d04-595566b6e9ff None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:12.468483 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a6b1b-7a1e-4e9d-9d04-595566b6e9ff None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:12.473257 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 316/635] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:12.473257 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a6b1b-7a1e-4e9d-9d04-595566b6e9ff None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:12.482854 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f950630-f662-4e6e-a14e-8dfc5299601f None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:12.515867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80164563-1343-4ab6-a3f6-808152532e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:12.516497 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80164563-1343-4ab6-a3f6-808152532e39 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:12.516777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80164563-1343-4ab6-a3f6-808152532e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:12.517078 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80164563-1343-4ab6-a3f6-808152532e39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:12.523523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80164563-1343-4ab6-a3f6-808152532e39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:12.525005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80164563-1343-4ab6-a3f6-808152532e39 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:12.536070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-80164563-1343-4ab6-a3f6-808152532e39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:12.536070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:12.536070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:12.536070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:12.536070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:12.536070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:12.536070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:12.536070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:12.536070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:12.536070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:12.536070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:12.536070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:12.536070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:12.536070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:12.536070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:12.537169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:12.537169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:12.537169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:12.537169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:12.537169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:12.537169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:12.537169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:12.537169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:12.537169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:12.537169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:12.537169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:12.537169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:12.537169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:12.537169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:12.537169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:12.537169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:12.537169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:12.538427 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:12.538427 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:12.538427 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:12.538427 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:12.538427 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:12.538427 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:12.538427 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:12.538427 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:12.538427 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 320/636] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:12.539032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c5783-48cc-4c62-88c3-9ce5dd5b8d3c None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:12.539032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c5783-48cc-4c62-88c3-9ce5dd5b8d3c None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:12.539191 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c5783-48cc-4c62-88c3-9ce5dd5b8d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:12.556717 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-c78c5783-48cc-4c62-88c3-9ce5dd5b8d3c None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:12.557897 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-c78c5783-48cc-4c62-88c3-9ce5dd5b8d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:12.569572 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:12.576865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a6b1b-7a1e-4e9d-9d04-595566b6e9ff None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:12.579365 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:12.613813 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258c683-dc17-480a-bddb-66fb37271bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:12.614419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258c683-dc17-480a-bddb-66fb37271bbb None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:12.614697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258c683-dc17-480a-bddb-66fb37271bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:12.615012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258c683-dc17-480a-bddb-66fb37271bbb None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:12.615746 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258c683-dc17-480a-bddb-66fb37271bbb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:12.616047 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258c683-dc17-480a-bddb-66fb37271bbb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:12.617508 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258c683-dc17-480a-bddb-66fb37271bbb None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:12.629566 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 317/637] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:12.634764 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 321/638] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:23:12.634924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258c683-dc17-480a-bddb-66fb37271bbb None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:12.635088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0332f49-b3cb-4bc9-b11a-7a462869b605 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:12.635088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0332f49-b3cb-4bc9-b11a-7a462869b605 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:12.635088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0332f49-b3cb-4bc9-b11a-7a462869b605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:12.658822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-c0332f49-b3cb-4bc9-b11a-7a462869b605 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:12.659991 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-c0332f49-b3cb-4bc9-b11a-7a462869b605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:12.726560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 322/639] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:23:12.727383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0332f49-b3cb-4bc9-b11a-7a462869b605 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:12.732951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258c683-dc17-480a-bddb-66fb37271bbb None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:12.767711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb0af5-9431-4c53-b43c-79af6e4c6965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:12.768353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb0af5-9431-4c53-b43c-79af6e4c6965 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:12.768627 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb0af5-9431-4c53-b43c-79af6e4c6965 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:12.768910 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb0af5-9431-4c53-b43c-79af6e4c6965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:12.780806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb0af5-9431-4c53-b43c-79af6e4c6965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:12.782278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb0af5-9431-4c53-b43c-79af6e4c6965 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:12.796579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-03bb0af5-9431-4c53-b43c-79af6e4c6965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:12.796579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:12.796579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:12.796579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:12.796579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:12.796579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:12.796579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:12.796579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:12.796579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:12.796579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:12.796579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:12.796579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:12.796579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:12.796579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:12.796579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:12.797482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:12.797482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:12.797482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:12.797482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:12.797482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:12.797482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:12.797482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:12.797482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:12.797482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:12.797482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:12.797482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:12.797482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:12.797482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:12.797482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:12.797482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:12.797482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:12.797482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:12.798356 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:12.798356 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:12.798356 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:12.798356 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:12.798356 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 318/640] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:12.798625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb0af5-9431-4c53-b43c-79af6e4c6965 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:12.809284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:12.822943 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:12.838563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0332f49-b3cb-4bc9-b11a-7a462869b605 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:12.871686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec7265-2fba-4628-908f-eed52556f593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:12.872346 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec7265-2fba-4628-908f-eed52556f593 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:12.872637 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec7265-2fba-4628-908f-eed52556f593 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:12.872917 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec7265-2fba-4628-908f-eed52556f593 None admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:12.873746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec7265-2fba-4628-908f-eed52556f593 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:12.874073 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec7265-2fba-4628-908f-eed52556f593 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:12.875643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec7265-2fba-4628-908f-eed52556f593 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:12.887530 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 323/641] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:12.888314 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec7265-2fba-4628-908f-eed52556f593 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:12.899530 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb0af5-9431-4c53-b43c-79af6e4c6965 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:12.933675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bda53f-28f2-45f4-8d7e-2dc085ab672f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:12.934333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bda53f-28f2-45f4-8d7e-2dc085ab672f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:12.934616 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bda53f-28f2-45f4-8d7e-2dc085ab672f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:12.934904 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bda53f-28f2-45f4-8d7e-2dc085ab672f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:12.935688 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bda53f-28f2-45f4-8d7e-2dc085ab672f None admin] RBAC: Adding query filter params (role.id=2aa0cdfed849496db8f3692c9569d7ee, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=56dfafd15e284b7986ba07dd3819bf65) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:12.935998 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bda53f-28f2-45f4-8d7e-2dc085ab672f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:12.937466 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bda53f-28f2-45f4-8d7e-2dc085ab672f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:12.943265 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 319/642] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:23:12 2022] GET /identity/v3/role_assignments?user.id=56dfafd15e284b7986ba07dd3819bf65&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=2aa0cdfed849496db8f3692c9569d7ee => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:12.944045 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bda53f-28f2-45f4-8d7e-2dc085ab672f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:12.994474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec7265-2fba-4628-908f-eed52556f593 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:13.021433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:13.037997 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa67166-7546-49cc-b242-577113ec404e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:13.038663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa67166-7546-49cc-b242-577113ec404e None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:13.038948 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa67166-7546-49cc-b242-577113ec404e None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:13.039319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa67166-7546-49cc-b242-577113ec404e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:13.051285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa67166-7546-49cc-b242-577113ec404e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:13.055796 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bda53f-28f2-45f4-8d7e-2dc085ab672f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:13.056904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-dfa67166-7546-49cc-b242-577113ec404e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:13.056904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:13.056904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:13.056904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:13.056904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:13.056904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:13.056904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:13.056904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:13.056904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:13.056904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:13.056904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:13.056904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:13.056904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:13.056904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:13.056904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:13.056904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:13.057880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:13.057880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:13.057880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:13.057880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:13.057880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:13.057880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:13.057880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:13.057880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:13.057880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:13.057880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:13.057880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:13.057880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:13.057880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:13.057880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:13.057880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:13.057880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:13.058792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:13.058792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:13.058792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:13.058792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:13.058792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:13.058792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:13.058792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:13.058792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:13.058792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:13.058792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:13.058792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:13.058792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:13.058792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 324/643] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:23:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:13.059546 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa67166-7546-49cc-b242-577113ec404e None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:13.073204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:13.100964 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3064f-293d-408a-bcaa-e3eb20ee158f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:13.101871 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3064f-293d-408a-bcaa-e3eb20ee158f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:13.101871 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3064f-293d-408a-bcaa-e3eb20ee158f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:13.102155 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3064f-293d-408a-bcaa-e3eb20ee158f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:13.116378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f3064f-293d-408a-bcaa-e3eb20ee158f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:13.122642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-c0f3064f-293d-408a-bcaa-e3eb20ee158f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:13.122642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:13.122642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:13.122642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:13.122642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:13.122642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:13.122642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:13.122642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:13.122642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:13.122642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:13.122642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:13.122642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:13.122642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:13.122642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:13.122642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:13.122642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:13.123573 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:13.123573 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:13.123573 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:13.123573 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:13.123573 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:13.123573 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:13.123573 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:13.123573 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:13.123573 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:13.123573 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:13.123573 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:13.123573 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:13.123573 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:13.123573 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:13.123573 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:13.123573 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:13.124935 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:13.124935 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:13.124935 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:13.124935 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:13.124935 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:13.124935 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:13.124935 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:13.124935 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:13.124935 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:13.124935 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:13.124935 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:13.124935 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:13.124935 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 320/644] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:23:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:13.124935 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3064f-293d-408a-bcaa-e3eb20ee158f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:13.186731 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa67166-7546-49cc-b242-577113ec404e None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:13.227858 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e6a76-f92a-4efc-b98c-5f41ca0db9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:13.228516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e6a76-f92a-4efc-b98c-5f41ca0db9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:13.228792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e6a76-f92a-4efc-b98c-5f41ca0db9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:13.229094 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e6a76-f92a-4efc-b98c-5f41ca0db9e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:13.229781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e6a76-f92a-4efc-b98c-5f41ca0db9e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:13.230732 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e6a76-f92a-4efc-b98c-5f41ca0db9e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:13.231988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e6a76-f92a-4efc-b98c-5f41ca0db9e7 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:13.239389 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3064f-293d-408a-bcaa-e3eb20ee158f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:13.244256 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 325/645] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:13.249056 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e6a76-f92a-4efc-b98c-5f41ca0db9e7 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:13.275869 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:13.282433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa35608-8404-4ccf-84d6-d76a3bc1d814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:13.283186 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa35608-8404-4ccf-84d6-d76a3bc1d814 None admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:13.283534 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa35608-8404-4ccf-84d6-d76a3bc1d814 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:13.283821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa35608-8404-4ccf-84d6-d76a3bc1d814 None admin] PATH_INFO: `/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/bb45ded9f75e4fa2973b564f83bdfaf2/roles/2aa0cdfed849496db8f3692c9569d7ee` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:13.299266 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa35608-8404-4ccf-84d6-d76a3bc1d814 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d6bcdd09cf24a5c8003951db37020cf, user_id=bb45ded9f75e4fa2973b564f83bdfaf2, role_id=2aa0cdfed849496db8f3692c9569d7ee)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:13.300705 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa35608-8404-4ccf-84d6-d76a3bc1d814 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:13.314421 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 321/646] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:23:13 2022] PUT /identity/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/bb45ded9f75e4fa2973b564f83bdfaf2/roles/2aa0cdfed849496db8f3692c9569d7ee => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:23:13.315222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa35608-8404-4ccf-84d6-d76a3bc1d814 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:13.336341 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:13.397604 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e6a76-f92a-4efc-b98c-5f41ca0db9e7 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:13.433887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa35608-8404-4ccf-84d6-d76a3bc1d814 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:13.452871 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5503719-99eb-4761-bc8f-86fc7cc46a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:13.454475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5503719-99eb-4761-bc8f-86fc7cc46a91 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:13.455016 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5503719-99eb-4761-bc8f-86fc7cc46a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:13.455503 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5503719-99eb-4761-bc8f-86fc7cc46a91 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:13.456496 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5503719-99eb-4761-bc8f-86fc7cc46a91 None admin] RBAC: Adding query filter params (name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:13.457675 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5503719-99eb-4761-bc8f-86fc7cc46a91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:13.459620 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5503719-99eb-4761-bc8f-86fc7cc46a91 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:13.472844 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 326/647] 10.222.0.74 () {56 vars in 1103 bytes} [Wed Jul 27 16:23:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:13.473683 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e2936-39c5-41aa-86b9-6fb1b484ad45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:13.474307 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e2936-39c5-41aa-86b9-6fb1b484ad45 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:13.474582 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e2936-39c5-41aa-86b9-6fb1b484ad45 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:13.474848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e2936-39c5-41aa-86b9-6fb1b484ad45 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:13.475531 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5503719-99eb-4761-bc8f-86fc7cc46a91 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:13.475639 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e2936-39c5-41aa-86b9-6fb1b484ad45 None admin] RBAC: Adding query filter params (name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:13.475862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e2936-39c5-41aa-86b9-6fb1b484ad45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:13.477441 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e2936-39c5-41aa-86b9-6fb1b484ad45 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:13.489652 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 322/648] 10.222.0.74 () {56 vars in 1103 bytes} [Wed Jul 27 16:23:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:13.491121 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e2936-39c5-41aa-86b9-6fb1b484ad45 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:13.546215 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:13.611615 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5503719-99eb-4761-bc8f-86fc7cc46a91 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:13.625171 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e2936-39c5-41aa-86b9-6fb1b484ad45 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:13.642805 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:13.652793 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52d722-3717-4fe5-80a6-8bb31d5418a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:13.653498 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52d722-3717-4fe5-80a6-8bb31d5418a6 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:13.653781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52d722-3717-4fe5-80a6-8bb31d5418a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:13.654061 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52d722-3717-4fe5-80a6-8bb31d5418a6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:13.662476 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52d722-3717-4fe5-80a6-8bb31d5418a6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:13.663939 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52d722-3717-4fe5-80a6-8bb31d5418a6 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:13.670886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805dd5a2-d6b8-44e6-85d9-baf7d0005349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:13.672066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805dd5a2-d6b8-44e6-85d9-baf7d0005349 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:13.672567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805dd5a2-d6b8-44e6-85d9-baf7d0005349 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:13.673063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805dd5a2-d6b8-44e6-85d9-baf7d0005349 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:13.676599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-fe52d722-3717-4fe5-80a6-8bb31d5418a6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 16:23:13.676599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:13.676599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:13.676599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:13.676599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:13.676599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:13.676599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:13.676599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:13.676599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:13.676599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:13.676599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:13.676599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:13.676599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:13.676599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:13.676599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:13.676599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:13.677545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:13.677545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:13.677545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:13.677545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:13.677545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:13.677545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:13.677545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:13.677545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:13.677545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:13.677545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:13.677545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:13.677545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:13.677545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:13.677545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:13.677545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:13.677545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:13.677545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:13.677545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:13.678486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:13.678486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:13.678486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:13.678486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:13.678486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:13.678486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:13.678486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:13.678486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:13.678486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 16:23:13.678486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:13.682541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 327/649] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:23:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:13.683607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52d722-3717-4fe5-80a6-8bb31d5418a6 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:13.689079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805dd5a2-d6b8-44e6-85d9-baf7d0005349 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:13.690559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805dd5a2-d6b8-44e6-85d9-baf7d0005349 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:13.710021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-805dd5a2-d6b8-44e6-85d9-baf7d0005349 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:13.710021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:13.710021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:13.710021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:13.710021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:13.710021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:13.710021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:13.710021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:13.710021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:13.710021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:13.710021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:13.710021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:13.710021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:13.710021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:13.710021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:13.710921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:13.710921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:13.710921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:13.710921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:13.710921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:13.710921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:13.710921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:13.710921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:13.710921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:13.710921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:13.710921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:13.710921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:13.710921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:13.710921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:13.710921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:13.710921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:13.710921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:13.711792 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:13.711792 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:13.711792 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:13.711792 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:13.716767 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 323/650] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:13.717833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805dd5a2-d6b8-44e6-85d9-baf7d0005349 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:13.808152 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52d722-3717-4fe5-80a6-8bb31d5418a6 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:13.843529 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee63f3a-f2fb-4891-a90b-9dd1b15954cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:13.844168 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee63f3a-f2fb-4891-a90b-9dd1b15954cc None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:13.844445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee63f3a-f2fb-4891-a90b-9dd1b15954cc None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:13.844716 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee63f3a-f2fb-4891-a90b-9dd1b15954cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:13.850135 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:13.858509 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee63f3a-f2fb-4891-a90b-9dd1b15954cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:13.859760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805dd5a2-d6b8-44e6-85d9-baf7d0005349 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:13.859957 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee63f3a-f2fb-4891-a90b-9dd1b15954cc None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:13.878230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-8ee63f3a-f2fb-4891-a90b-9dd1b15954cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:13.878230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:13.878230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:13.878230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:13.878230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:13.878230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:13.878230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:13.878230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:13.878230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:13.878230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:13.878230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:13.878230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:13.878230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:13.878230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:13.878230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:13.879311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:13.879311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:13.879311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:13.879311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:13.879311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:13.879311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:13.879311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:13.879311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:13.879311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:13.879311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:13.879311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:13.879311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:13.879311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:13.879311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:13.879311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:13.879311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:13.879311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:13.880223 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:13.880223 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:13.880223 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:13.880223 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:13.880223 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 328/651] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:13.880495 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee63f3a-f2fb-4891-a90b-9dd1b15954cc None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:13.909168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be706b37-78a9-43eb-b0a3-245dc7f80180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:13.909865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be706b37-78a9-43eb-b0a3-245dc7f80180 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:13.910144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be706b37-78a9-43eb-b0a3-245dc7f80180 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:13.910463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be706b37-78a9-43eb-b0a3-245dc7f80180 None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:13.911299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be706b37-78a9-43eb-b0a3-245dc7f80180 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:13.911614 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be706b37-78a9-43eb-b0a3-245dc7f80180 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:13.913063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be706b37-78a9-43eb-b0a3-245dc7f80180 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:13.931837 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 324/652] 10.222.0.74 () {56 vars in 1137 bytes} [Wed Jul 27 16:23:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:13.932815 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be706b37-78a9-43eb-b0a3-245dc7f80180 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:13.934031 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:14.028502 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee63f3a-f2fb-4891-a90b-9dd1b15954cc None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:14.034788 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be706b37-78a9-43eb-b0a3-245dc7f80180 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:14.067971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30077c-9021-4c11-9f40-fefc89e1fe81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:14.069232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30077c-9021-4c11-9f40-fefc89e1fe81 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:14.069792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30077c-9021-4c11-9f40-fefc89e1fe81 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:14.070313 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30077c-9021-4c11-9f40-fefc89e1fe81 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:14.071506 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30077c-9021-4c11-9f40-fefc89e1fe81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:14.071991 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30077c-9021-4c11-9f40-fefc89e1fe81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:14.072097 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce55e9-620c-4dfa-8446-227955df4f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:14.072787 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce55e9-620c-4dfa-8446-227955df4f7b None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:14.073181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce55e9-620c-4dfa-8446-227955df4f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:14.074064 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce55e9-620c-4dfa-8446-227955df4f7b None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:14.074875 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30077c-9021-4c11-9f40-fefc89e1fe81 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:14.075058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce55e9-620c-4dfa-8446-227955df4f7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:14.075729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce55e9-620c-4dfa-8446-227955df4f7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:14.077203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce55e9-620c-4dfa-8446-227955df4f7b None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:14.093069 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 325/654] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:14.093069 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce55e9-620c-4dfa-8446-227955df4f7b None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:14.093357 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 329/654] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:14.095531 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30077c-9021-4c11-9f40-fefc89e1fe81 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:14.111154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:14.220252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:14.224429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30077c-9021-4c11-9f40-fefc89e1fe81 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:14.237605 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce55e9-620c-4dfa-8446-227955df4f7b None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:14.263392 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de88f08-77be-48a9-aa12-397fa5c8ea6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:14.264035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de88f08-77be-48a9-aa12-397fa5c8ea6d None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:14.264317 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de88f08-77be-48a9-aa12-397fa5c8ea6d None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:14.264594 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de88f08-77be-48a9-aa12-397fa5c8ea6d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:14.271911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de88f08-77be-48a9-aa12-397fa5c8ea6d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:14.273406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de88f08-77be-48a9-aa12-397fa5c8ea6d None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:14.283582 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8047c97-6314-4eda-9535-3dda198299b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:14.284463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8047c97-6314-4eda-9535-3dda198299b3 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:14.284974 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8047c97-6314-4eda-9535-3dda198299b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:14.285452 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8047c97-6314-4eda-9535-3dda198299b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:14.286134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-3de88f08-77be-48a9-aa12-397fa5c8ea6d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 16:23:14.286134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:14.286134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:14.286134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:14.286134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:14.286134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:14.286134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:14.286134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:14.286134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:14.286134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:14.286134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:14.286134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:14.286134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:14.286134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:14.286134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:14.286134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:14.287043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:14.287043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:14.287043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:14.287043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:14.287043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:14.287043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:14.287043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:14.287043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:14.287043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:14.287043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:14.287043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:14.287043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:14.287043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:14.287043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:14.287043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:14.287043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:14.287043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:14.287043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:14.287988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:14.287988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:14.287988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:14.287988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:14.287988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:14.287988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:14.287988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:14.287988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:14.287988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 16:23:14.287988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:14.287988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 330/655] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:23:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:14.288574 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de88f08-77be-48a9-aa12-397fa5c8ea6d None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:14.370709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8047c97-6314-4eda-9535-3dda198299b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:14.372502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8047c97-6314-4eda-9535-3dda198299b3 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:14.388322 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:14.399343 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de88f08-77be-48a9-aa12-397fa5c8ea6d None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:14.434870 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe734332-a494-46d0-bc1e-f0cde035f995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:14.435522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe734332-a494-46d0-bc1e-f0cde035f995 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:14.435875 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe734332-a494-46d0-bc1e-f0cde035f995 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:14.436224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe734332-a494-46d0-bc1e-f0cde035f995 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:14.444241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe734332-a494-46d0-bc1e-f0cde035f995 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:14.445802 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe734332-a494-46d0-bc1e-f0cde035f995 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:14.457837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-fe734332-a494-46d0-bc1e-f0cde035f995 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 16:23:14.457837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:14.457837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:14.457837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:14.457837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:14.457837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:14.457837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:14.457837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:14.457837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:14.457837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:14.457837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:14.457837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:14.457837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:14.457837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:14.457837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:14.457837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:14.458788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:14.458788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:14.458788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:14.458788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:14.458788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:14.458788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:14.458788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:14.458788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:14.458788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:14.458788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:14.458788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:14.458788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:14.458788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:14.458788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:14.458788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:14.458788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:14.458788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:14.458788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:14.459707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:14.459707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:14.459707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:14.459707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:14.459707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:14.459707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:14.459707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:14.459707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:14.459707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 16:23:14.459707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:14.463805 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 331/656] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:23:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:14.465276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe734332-a494-46d0-bc1e-f0cde035f995 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:14.489860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:14.491974 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 326/657] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:23:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:23:14.493638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8047c97-6314-4eda-9535-3dda198299b3 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:14.572167 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe734332-a494-46d0-bc1e-f0cde035f995 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:14.604909 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8047c97-6314-4eda-9535-3dda198299b3 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:14.608387 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d938400-f7e9-4a5a-b0b6-e89584040cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:14.608651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d938400-f7e9-4a5a-b0b6-e89584040cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:14.608959 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d938400-f7e9-4a5a-b0b6-e89584040cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:14.609238 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d938400-f7e9-4a5a-b0b6-e89584040cc8 None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:14.610023 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d938400-f7e9-4a5a-b0b6-e89584040cc8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:14.610328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d938400-f7e9-4a5a-b0b6-e89584040cc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:14.611751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d938400-f7e9-4a5a-b0b6-e89584040cc8 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:14.622547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:14.639809 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 332/658] 10.222.0.74 () {56 vars in 1139 bytes} [Wed Jul 27 16:23:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:14.640677 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d938400-f7e9-4a5a-b0b6-e89584040cc8 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:14.649605 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb4586-ab30-4330-9060-c9773f27c2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:14.650539 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb4586-ab30-4330-9060-c9773f27c2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:14.651085 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb4586-ab30-4330-9060-c9773f27c2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:14.652057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb4586-ab30-4330-9060-c9773f27c2d8 None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:14.653615 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb4586-ab30-4330-9060-c9773f27c2d8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:14.654130 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb4586-ab30-4330-9060-c9773f27c2d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:14.655763 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb4586-ab30-4330-9060-c9773f27c2d8 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:14.675853 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 327/659] 10.222.0.74 () {56 vars in 1137 bytes} [Wed Jul 27 16:23:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:14.677133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb4586-ab30-4330-9060-c9773f27c2d8 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:14.733137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:14.758333 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d938400-f7e9-4a5a-b0b6-e89584040cc8 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:14.783870 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb4586-ab30-4330-9060-c9773f27c2d8 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:14.796818 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980acf15-7ebf-4c99-8224-f96f1227c695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:14.797504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980acf15-7ebf-4c99-8224-f96f1227c695 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:14.797784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980acf15-7ebf-4c99-8224-f96f1227c695 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:14.798109 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980acf15-7ebf-4c99-8224-f96f1227c695 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:14.809308 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980acf15-7ebf-4c99-8224-f96f1227c695 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:14.810856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980acf15-7ebf-4c99-8224-f96f1227c695 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:14.820827 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa41020-2991-4c79-a02c-e1f21d096d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:14.821756 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa41020-2991-4c79-a02c-e1f21d096d3c None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:14.822261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa41020-2991-4c79-a02c-e1f21d096d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:14.822738 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa41020-2991-4c79-a02c-e1f21d096d3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:14.828136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-980acf15-7ebf-4c99-8224-f96f1227c695 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:14.828136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:14.828136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:14.828136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:14.828136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:14.828136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:14.828136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:14.828136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:14.828136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:14.828136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:14.828136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:14.828136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:14.828136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:14.828136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:14.828136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:14.829028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:14.829028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:14.829028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:14.829028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:14.829028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:14.829028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:14.829028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:14.829028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:14.829028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:14.829028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:14.829028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:14.829028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:14.829028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:14.829028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:14.829028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:14.829028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:14.829028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:14.829937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:14.829937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:14.829937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:14.829937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:14.829937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 333/660] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:14.830617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980acf15-7ebf-4c99-8224-f96f1227c695 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:14.870656 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:14.900355 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa41020-2991-4c79-a02c-e1f21d096d3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:14.902249 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa41020-2991-4c79-a02c-e1f21d096d3c None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:14.945013 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980acf15-7ebf-4c99-8224-f96f1227c695 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:14.977491 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:14.985986 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a951d1c-1198-43d4-90fa-d6effeffd3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:14.986558 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a951d1c-1198-43d4-90fa-d6effeffd3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:14.986840 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a951d1c-1198-43d4-90fa-d6effeffd3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:14.987116 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a951d1c-1198-43d4-90fa-d6effeffd3d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:15.008431 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 328/661] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:23:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:23:15.009669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa41020-2991-4c79-a02c-e1f21d096d3c None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:15.063721 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a951d1c-1198-43d4-90fa-d6effeffd3d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:15.065447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a951d1c-1198-43d4-90fa-d6effeffd3d8 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:15.083765 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:15.115204 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa41020-2991-4c79-a02c-e1f21d096d3c None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:15.152520 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e22b90-a810-424e-b5fe-57c6c4d5f2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:15.153842 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e22b90-a810-424e-b5fe-57c6c4d5f2f7 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:15.154540 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e22b90-a810-424e-b5fe-57c6c4d5f2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:15.154982 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e22b90-a810-424e-b5fe-57c6c4d5f2f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:15.155684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e22b90-a810-424e-b5fe-57c6c4d5f2f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:15.156004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e22b90-a810-424e-b5fe-57c6c4d5f2f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:15.157418 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e22b90-a810-424e-b5fe-57c6c4d5f2f7 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:15.175387 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 329/662] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:15.175387 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e22b90-a810-424e-b5fe-57c6c4d5f2f7 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:15.185062 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 334/663] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:23:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:23:15.188924 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa652c75-323a-458d-a983-6c3849012339 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:15.188924 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa652c75-323a-458d-a983-6c3849012339 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:15.188924 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa652c75-323a-458d-a983-6c3849012339 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:15.188924 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 335/664] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:15.188924 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa652c75-323a-458d-a983-6c3849012339 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:15.222202 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:15.282237 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e22b90-a810-424e-b5fe-57c6c4d5f2f7 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:15.301084 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa652c75-323a-458d-a983-6c3849012339 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:15.313533 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:15.319402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff0a0c-5659-4cef-9ba3-eb7e459eeb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:15.320051 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff0a0c-5659-4cef-9ba3-eb7e459eeb67 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:15.320334 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff0a0c-5659-4cef-9ba3-eb7e459eeb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:15.320611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff0a0c-5659-4cef-9ba3-eb7e459eeb67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:15.332678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff0a0c-5659-4cef-9ba3-eb7e459eeb67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:15.334352 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff0a0c-5659-4cef-9ba3-eb7e459eeb67 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:15.336878 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592723a-2eab-4fb8-82c0-0d9782c0eeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:15.337546 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592723a-2eab-4fb8-82c0-0d9782c0eeeb None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:15.337823 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592723a-2eab-4fb8-82c0-0d9782c0eeeb None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:15.338099 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592723a-2eab-4fb8-82c0-0d9782c0eeeb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:15.344904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592723a-2eab-4fb8-82c0-0d9782c0eeeb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:15.346357 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592723a-2eab-4fb8-82c0-0d9782c0eeeb None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:15.349471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-51ff0a0c-5659-4cef-9ba3-eb7e459eeb67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:15.349471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:15.349471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:15.349471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:15.349471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:15.349471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:15.349471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:15.349471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:15.349471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:15.349471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:15.349471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:15.349471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:15.349471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:15.349471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:15.349471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:15.350320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:15.350320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:15.350320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:15.350320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:15.350320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:15.350320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:15.350320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:15.350320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:15.350320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:15.350320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:15.350320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:15.350320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:15.350320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:15.350320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:15.350320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:15.350320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:15.350320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:15.351213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:15.351213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:15.351213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:15.351213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:15.355309 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 330/665] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:15.356179 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff0a0c-5659-4cef-9ba3-eb7e459eeb67 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:15.357740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-0592723a-2eab-4fb8-82c0-0d9782c0eeeb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:15.357740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:15.357740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:15.357740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:15.357740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:15.357740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:15.357740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:15.357740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:15.357740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:15.357740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:15.357740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:15.357740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:15.357740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:15.357740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:15.357740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:15.358560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:15.358560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:15.358560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:15.358560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:15.358560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:15.358560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:15.358560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:15.358560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:15.358560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:15.358560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:15.358560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:15.358560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:15.358560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:15.358560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:15.358560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:15.358560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:15.358560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:15.359430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:15.359430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:15.359430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:15.359430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:15.359430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:15.359430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:15.359430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:15.359430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:15.359430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 336/666] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:15.360960 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca69ec-1295-4a0b-8eb3-4be98e1b2d6f None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:15.360960 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca69ec-1295-4a0b-8eb3-4be98e1b2d6f None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:15.360960 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca69ec-1295-4a0b-8eb3-4be98e1b2d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:15.378273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-9cca69ec-1295-4a0b-8eb3-4be98e1b2d6f None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:15.379454 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-9cca69ec-1295-4a0b-8eb3-4be98e1b2d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:15.446016 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 337/667] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:23:15.446859 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca69ec-1295-4a0b-8eb3-4be98e1b2d6f None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:15.463051 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff0a0c-5659-4cef-9ba3-eb7e459eeb67 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:15.481623 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:15.497361 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f60b6-8092-46e6-b95e-d8ef835f279b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:15.498005 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f60b6-8092-46e6-b95e-d8ef835f279b None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:15.498286 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f60b6-8092-46e6-b95e-d8ef835f279b None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:15.498564 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f60b6-8092-46e6-b95e-d8ef835f279b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:15.509011 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f60b6-8092-46e6-b95e-d8ef835f279b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:15.510462 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f60b6-8092-46e6-b95e-d8ef835f279b None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:15.525242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-1e3f60b6-8092-46e6-b95e-d8ef835f279b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:15.525242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:15.525242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:15.525242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:15.525242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:15.525242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:15.525242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:15.525242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:15.525242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:15.525242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:15.525242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:15.525242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:15.525242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:15.525242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:15.525242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:15.526297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:15.526297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:15.526297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:15.526297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:15.526297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:15.526297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:15.526297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:15.526297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:15.526297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:15.526297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:15.526297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:15.526297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:15.526297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:15.526297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:15.526297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:15.526297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:15.526297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:15.527290 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:15.527290 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:15.527290 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:15.527290 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:15.527290 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 331/668] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:15.527290 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f60b6-8092-46e6-b95e-d8ef835f279b None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:15.560685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca69ec-1295-4a0b-8eb3-4be98e1b2d6f None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:15.563932 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:15.595711 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435657b-38f9-4d92-8c87-3f77a9b51657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:15.596330 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435657b-38f9-4d92-8c87-3f77a9b51657 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:15.596615 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435657b-38f9-4d92-8c87-3f77a9b51657 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:15.596915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435657b-38f9-4d92-8c87-3f77a9b51657 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:15.597607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3435657b-38f9-4d92-8c87-3f77a9b51657 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:15.597914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3435657b-38f9-4d92-8c87-3f77a9b51657 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:15.599291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3435657b-38f9-4d92-8c87-3f77a9b51657 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:15.610380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 338/669] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:15.611547 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d977d-f86d-4350-b4d2-3715da8a0f6b None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:15.611833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d977d-f86d-4350-b4d2-3715da8a0f6b None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:15.612114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d977d-f86d-4350-b4d2-3715da8a0f6b None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:15.613356 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 339/670] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:15.614053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d977d-f86d-4350-b4d2-3715da8a0f6b None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:15.633347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f60b6-8092-46e6-b95e-d8ef835f279b None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:15.666621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0881c-7ff9-4786-bf5f-ad41ccc514ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:15.667240 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0881c-7ff9-4786-bf5f-ad41ccc514ef None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:15.667528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0881c-7ff9-4786-bf5f-ad41ccc514ef None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:15.667809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0881c-7ff9-4786-bf5f-ad41ccc514ef None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:15.668556 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0881c-7ff9-4786-bf5f-ad41ccc514ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:15.668864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0881c-7ff9-4786-bf5f-ad41ccc514ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:15.670323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0881c-7ff9-4786-bf5f-ad41ccc514ef None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:15.682623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 332/671] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:15.683439 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0881c-7ff9-4786-bf5f-ad41ccc514ef None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:15.703822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:15.724212 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d977d-f86d-4350-b4d2-3715da8a0f6b None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:15.757491 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47dda14-55e3-49f2-8a39-422f5c23a8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:15.758205 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47dda14-55e3-49f2-8a39-422f5c23a8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:15.758556 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47dda14-55e3-49f2-8a39-422f5c23a8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:15.758895 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47dda14-55e3-49f2-8a39-422f5c23a8a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:15.769485 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47dda14-55e3-49f2-8a39-422f5c23a8a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:15.774781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-c47dda14-55e3-49f2-8a39-422f5c23a8a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:23:15.774781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:15.774781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:15.774781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:15.774781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:15.774781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:15.774781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:15.774781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:15.774781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:15.774781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:15.774781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:15.774781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:15.774781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:15.774781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:15.774781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:15.774781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:15.775857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:15.775857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:15.775857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:15.775857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:15.775857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:15.775857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:15.775857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:15.775857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:15.775857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:15.775857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:15.775857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:15.775857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:15.775857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:15.775857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:15.775857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:15.775857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:15.776841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:15.776841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:15.776841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:15.776841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:15.776841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:15.776841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:15.776841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:15.776841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:15.776841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:15.776841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:15.776841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:23:15.776841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:15.778848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:15.780829 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 340/672] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:23:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:15.781708 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47dda14-55e3-49f2-8a39-422f5c23a8a3 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:15.783208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0881c-7ff9-4786-bf5f-ad41ccc514ef None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:15.819775 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff652c8-ebe0-429a-ba06-8aad27f36e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:15.820318 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff652c8-ebe0-429a-ba06-8aad27f36e43 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:15.820597 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff652c8-ebe0-429a-ba06-8aad27f36e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:15.820876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff652c8-ebe0-429a-ba06-8aad27f36e43 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:15.821585 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff652c8-ebe0-429a-ba06-8aad27f36e43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:15.821885 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff652c8-ebe0-429a-ba06-8aad27f36e43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:15.823252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff652c8-ebe0-429a-ba06-8aad27f36e43 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:15.834528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 333/673] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:15.835661 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd1324-b96e-487c-8c83-b7422108acfb None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:15.835946 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd1324-b96e-487c-8c83-b7422108acfb None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:15.836223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd1324-b96e-487c-8c83-b7422108acfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:15.853491 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-56bd1324-b96e-487c-8c83-b7422108acfb None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:15.854673 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-56bd1324-b96e-487c-8c83-b7422108acfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:15.880249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47dda14-55e3-49f2-8a39-422f5c23a8a3 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:15.913377 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5222c06-1110-40f8-beb5-1137a93f0b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:15.914008 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5222c06-1110-40f8-beb5-1137a93f0b49 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:15.914287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5222c06-1110-40f8-beb5-1137a93f0b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:15.914569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5222c06-1110-40f8-beb5-1137a93f0b49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:15.921291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 334/674] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:23:15.921591 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5222c06-1110-40f8-beb5-1137a93f0b49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:15.922222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd1324-b96e-487c-8c83-b7422108acfb None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:15.923029 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5222c06-1110-40f8-beb5-1137a93f0b49 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:15.930843 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:15.945663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-d5222c06-1110-40f8-beb5-1137a93f0b49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:15.945663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:15.945663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:15.945663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:15.945663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:15.945663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:15.945663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:15.945663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:15.945663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:15.945663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:15.945663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:15.945663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:15.945663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:15.945663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:15.945663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:15.946881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:15.946881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:15.946881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:15.946881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:15.946881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:15.946881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:15.946881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:15.946881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:15.946881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:15.946881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:15.946881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:15.946881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:15.946881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:15.946881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:15.946881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:15.946881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:15.946881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:15.947753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:15.947753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:15.947753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:15.947753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:15.947753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:15.947753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:15.947753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:15.947753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:15.947753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 341/675] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:15.948279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5222c06-1110-40f8-beb5-1137a93f0b49 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:16.014012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:16.029324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd1324-b96e-487c-8c83-b7422108acfb None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:16.048692 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5222c06-1110-40f8-beb5-1137a93f0b49 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:16.062690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4551a7-9b56-44d9-8e30-dd4f9af62699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:16.063304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4551a7-9b56-44d9-8e30-dd4f9af62699 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:16.063580 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4551a7-9b56-44d9-8e30-dd4f9af62699 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:16.063850 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4551a7-9b56-44d9-8e30-dd4f9af62699 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:16.064631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4551a7-9b56-44d9-8e30-dd4f9af62699 None admin] RBAC: Adding query filter params (role.id=2aa0cdfed849496db8f3692c9569d7ee, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=c50c2d1781b04a5fa8be81bce9205747) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:16.064961 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4551a7-9b56-44d9-8e30-dd4f9af62699 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:16.067385 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4551a7-9b56-44d9-8e30-dd4f9af62699 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:16.073194 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 335/676] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:23:15 2022] GET /identity/v3/role_assignments?user.id=c50c2d1781b04a5fa8be81bce9205747&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=2aa0cdfed849496db8f3692c9569d7ee => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:16.073929 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4551a7-9b56-44d9-8e30-dd4f9af62699 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:16.096748 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfcd937-5bb7-428f-a383-7d5b6be3652f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:16.097503 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfcd937-5bb7-428f-a383-7d5b6be3652f None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:16.097854 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfcd937-5bb7-428f-a383-7d5b6be3652f None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:16.098189 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfcd937-5bb7-428f-a383-7d5b6be3652f None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:16.099085 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfcd937-5bb7-428f-a383-7d5b6be3652f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:16.099378 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfcd937-5bb7-428f-a383-7d5b6be3652f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:16.101212 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfcd937-5bb7-428f-a383-7d5b6be3652f None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:16.115851 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 342/677] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:23:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:16.115851 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfcd937-5bb7-428f-a383-7d5b6be3652f None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:16.192514 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:16.193234 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4551a7-9b56-44d9-8e30-dd4f9af62699 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:16.234110 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7eb086-728b-4a54-9f9c-02886c979099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:16.234763 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7eb086-728b-4a54-9f9c-02886c979099 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:16.235037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7eb086-728b-4a54-9f9c-02886c979099 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:16.235301 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfcd937-5bb7-428f-a383-7d5b6be3652f None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:16.235399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7eb086-728b-4a54-9f9c-02886c979099 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:16.242499 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7eb086-728b-4a54-9f9c-02886c979099 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:16.243960 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7eb086-728b-4a54-9f9c-02886c979099 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:16.256357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-bf7eb086-728b-4a54-9f9c-02886c979099 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:16.256357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:16.256357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:16.256357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:16.256357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:16.256357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:16.256357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:16.256357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:16.256357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:16.256357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:16.256357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:16.256357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:16.256357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:16.256357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:16.256357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:16.257631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:16.257631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:16.257631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:16.257631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:16.257631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:16.257631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:16.257631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:16.257631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:16.257631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:16.257631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:16.257631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:16.257631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:16.257631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:16.257631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:16.257631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:16.257631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:16.257631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:16.258504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:16.258504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:16.258504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:16.258504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:16.258504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:16.258504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:16.258504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:16.258504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:16.258504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 336/678] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:16.258504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7eb086-728b-4a54-9f9c-02886c979099 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:16.272041 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:16.280333 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90ec3b-cee3-404e-afe0-9f78b0d1fc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:16.281026 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90ec3b-cee3-404e-afe0-9f78b0d1fc5e None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:16.281309 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90ec3b-cee3-404e-afe0-9f78b0d1fc5e None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:16.281584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90ec3b-cee3-404e-afe0-9f78b0d1fc5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:16.293704 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90ec3b-cee3-404e-afe0-9f78b0d1fc5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:16.298849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-da90ec3b-cee3-404e-afe0-9f78b0d1fc5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:16.298849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:16.298849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:16.298849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:16.298849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:16.298849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:16.298849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:16.298849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:16.298849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:16.298849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:16.298849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:16.298849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:16.298849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:16.298849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:16.298849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:16.298849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:16.299777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:16.299777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:16.299777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:16.299777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:16.299777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:16.299777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:16.299777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:16.299777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:16.299777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:16.299777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:16.299777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:16.299777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:16.299777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:16.299777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:16.299777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:16.299777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:16.300602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:16.300602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:16.300602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:16.300602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:16.300602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:16.300602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:16.300602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:16.300602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:16.300602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:16.300602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:16.300602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:16.300602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:16.300602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 343/679] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:23:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:16.301316 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90ec3b-cee3-404e-afe0-9f78b0d1fc5e None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:16.378338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7eb086-728b-4a54-9f9c-02886c979099 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:16.415622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858e47a-137f-434b-9e7b-5ed4678ef13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:16.416235 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90ec3b-cee3-404e-afe0-9f78b0d1fc5e None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:16.416349 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858e47a-137f-434b-9e7b-5ed4678ef13b None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:16.416508 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858e47a-137f-434b-9e7b-5ed4678ef13b None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:16.416785 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858e47a-137f-434b-9e7b-5ed4678ef13b None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:16.418339 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858e47a-137f-434b-9e7b-5ed4678ef13b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:16.418653 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858e47a-137f-434b-9e7b-5ed4678ef13b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:16.421028 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858e47a-137f-434b-9e7b-5ed4678ef13b None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:16.444314 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 337/680] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:16.444527 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:16.444625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858e47a-137f-434b-9e7b-5ed4678ef13b None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:16.453486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6e729-e248-451b-bce6-9e610143751b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:16.454130 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6e729-e248-451b-bce6-9e610143751b None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:16.454408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6e729-e248-451b-bce6-9e610143751b None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:16.454676 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6e729-e248-451b-bce6-9e610143751b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:16.465829 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e6e729-e248-451b-bce6-9e610143751b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:16.467277 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e6e729-e248-451b-bce6-9e610143751b None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:16.481921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-95e6e729-e248-451b-bce6-9e610143751b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:16.481921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:16.481921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:16.481921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:16.481921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:16.481921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:16.481921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:16.481921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:16.481921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:16.481921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:16.481921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:16.481921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:16.481921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:16.481921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:16.481921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:16.483076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:16.483076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:16.483076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:16.483076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:16.483076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:16.483076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:16.483076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:16.483076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:16.483076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:16.483076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:16.483076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:16.483076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:16.483076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:16.483076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:16.483076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:16.483076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:16.483076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:16.483939 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:16.483939 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:16.483939 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:16.483939 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:16.483939 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 344/681] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:16.483939 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6e729-e248-451b-bce6-9e610143751b None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:16.527276 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:16.559513 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858e47a-137f-434b-9e7b-5ed4678ef13b None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:16.591107 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6e729-e248-451b-bce6-9e610143751b None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:16.593422 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ecb90-123f-409a-bcf8-76d136320cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:16.594112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ecb90-123f-409a-bcf8-76d136320cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:16.594491 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ecb90-123f-409a-bcf8-76d136320cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:16.594867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ecb90-123f-409a-bcf8-76d136320cb2 None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:16.595721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ecb90-123f-409a-bcf8-76d136320cb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:16.596123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ecb90-123f-409a-bcf8-76d136320cb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:16.597661 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ecb90-123f-409a-bcf8-76d136320cb2 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:16.610179 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 338/682] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:16.610179 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ecb90-123f-409a-bcf8-76d136320cb2 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:16.634166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee276a7-0e30-4404-86cd-44f890ba0d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:16.634808 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee276a7-0e30-4404-86cd-44f890ba0d18 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:16.635157 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee276a7-0e30-4404-86cd-44f890ba0d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:16.635520 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee276a7-0e30-4404-86cd-44f890ba0d18 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:16.636318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee276a7-0e30-4404-86cd-44f890ba0d18 None admin] RBAC: Adding query filter params (name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:16.636684 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee276a7-0e30-4404-86cd-44f890ba0d18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:16.639068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee276a7-0e30-4404-86cd-44f890ba0d18 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:16.655237 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 345/683] 10.222.0.74 () {56 vars in 1103 bytes} [Wed Jul 27 16:23:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:16.656254 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee276a7-0e30-4404-86cd-44f890ba0d18 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:16.702074 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:16.730449 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ecb90-123f-409a-bcf8-76d136320cb2 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:16.758020 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:16.764041 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa4829-1cf6-43ef-867a-a8da0067fdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:16.764654 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa4829-1cf6-43ef-867a-a8da0067fdf8 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:16.764956 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa4829-1cf6-43ef-867a-a8da0067fdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:16.765237 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa4829-1cf6-43ef-867a-a8da0067fdf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:16.766010 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa4829-1cf6-43ef-867a-a8da0067fdf8 None admin] RBAC: Adding query filter params (role.id=2aa0cdfed849496db8f3692c9569d7ee, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=193dd9fee1844275ad0dab602e93df11) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:16.766312 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa4829-1cf6-43ef-867a-a8da0067fdf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:16.767735 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa4829-1cf6-43ef-867a-a8da0067fdf8 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:16.773558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 339/684] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:23:16 2022] GET /identity/v3/role_assignments?user.id=193dd9fee1844275ad0dab602e93df11&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=2aa0cdfed849496db8f3692c9569d7ee => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:16.774340 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa4829-1cf6-43ef-867a-a8da0067fdf8 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:16.789864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee276a7-0e30-4404-86cd-44f890ba0d18 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:16.841948 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495005ee-f4ed-4d09-9c43-d33e649ea026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:16.843266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495005ee-f4ed-4d09-9c43-d33e649ea026 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:16.844237 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495005ee-f4ed-4d09-9c43-d33e649ea026 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:16.844908 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495005ee-f4ed-4d09-9c43-d33e649ea026 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:16.845946 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495005ee-f4ed-4d09-9c43-d33e649ea026 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:16.846441 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495005ee-f4ed-4d09-9c43-d33e649ea026 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:16.848331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495005ee-f4ed-4d09-9c43-d33e649ea026 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:16.871153 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 346/685] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:16.871153 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495005ee-f4ed-4d09-9c43-d33e649ea026 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:16.894065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa4829-1cf6-43ef-867a-a8da0067fdf8 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:16.940401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08480e-e551-4070-afce-29c0cbecf9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:16.941278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08480e-e551-4070-afce-29c0cbecf9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:16.941664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08480e-e551-4070-afce-29c0cbecf9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:16.942090 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08480e-e551-4070-afce-29c0cbecf9f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:16.942992 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08480e-e551-4070-afce-29c0cbecf9f2 None admin] RBAC: Adding query filter params (role.id=2aa0cdfed849496db8f3692c9569d7ee, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=dd352d423bff418f932b434c51b15d23) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:16.943408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08480e-e551-4070-afce-29c0cbecf9f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:16.945188 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08480e-e551-4070-afce-29c0cbecf9f2 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:16.955432 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 340/686] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:23:16 2022] GET /identity/v3/role_assignments?user.id=dd352d423bff418f932b434c51b15d23&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=2aa0cdfed849496db8f3692c9569d7ee => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:16.956918 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08480e-e551-4070-afce-29c0cbecf9f2 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:17.017392 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:17.029148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:17.030365 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495005ee-f4ed-4d09-9c43-d33e649ea026 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:17.077099 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8cfcd-553a-4fa3-a996-4b1f0da6d5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:17.078058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8cfcd-553a-4fa3-a996-4b1f0da6d5df None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:17.078503 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8cfcd-553a-4fa3-a996-4b1f0da6d5df None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:17.078942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8cfcd-553a-4fa3-a996-4b1f0da6d5df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:17.079943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08480e-e551-4070-afce-29c0cbecf9f2 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:17.092219 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8cfcd-553a-4fa3-a996-4b1f0da6d5df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:17.094011 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8cfcd-553a-4fa3-a996-4b1f0da6d5df None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:17.112856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-30b8cfcd-553a-4fa3-a996-4b1f0da6d5df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:17.112856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:17.112856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:17.112856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:17.112856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:17.112856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:17.112856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:17.112856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:17.112856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:17.112856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:17.112856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:17.112856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:17.112856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:17.112856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:17.112856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:17.114102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:17.114102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:17.114102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:17.114102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:17.114102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:17.114102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:17.114102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:17.114102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:17.114102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:17.114102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:17.114102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:17.114102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:17.114102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:17.114102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:17.114102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:17.114102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:17.114102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:17.115248 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:17.115248 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:17.115248 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:17.115248 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:17.117885 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 347/687] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:17.119261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc2225-f5a1-43e9-805e-ffd35b5ab8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:17.119556 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc2225-f5a1-43e9-805e-ffd35b5ab8dd None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:17.119906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc2225-f5a1-43e9-805e-ffd35b5ab8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:17.120248 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc2225-f5a1-43e9-805e-ffd35b5ab8dd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:17.124184 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8cfcd-553a-4fa3-a996-4b1f0da6d5df None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:17.132148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bc2225-f5a1-43e9-805e-ffd35b5ab8dd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:17.133756 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bc2225-f5a1-43e9-805e-ffd35b5ab8dd None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:17.146810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-e8bc2225-f5a1-43e9-805e-ffd35b5ab8dd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 16:23:17.146810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:17.146810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:17.146810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:17.146810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:17.146810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:17.146810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:17.146810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:17.146810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:17.146810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:17.146810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:17.146810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:17.146810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:17.146810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:17.146810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:17.146810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:17.147996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:17.147996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:17.147996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:17.147996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:17.147996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:17.147996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:17.147996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:17.147996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:17.147996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:17.147996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:17.147996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:17.147996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:17.147996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:17.147996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:17.147996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:17.147996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:17.147996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:17.147996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:17.150079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:17.150079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:17.150079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:17.150079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:17.150079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:17.150079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:17.150079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:17.150079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:17.150079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 16:23:17.150079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:17.150079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 341/688] 10.222.0.74 () {56 vars in 1093 bytes} [Wed Jul 27 16:23:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:17.155209 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc2225-f5a1-43e9-805e-ffd35b5ab8dd None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:17.262671 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8cfcd-553a-4fa3-a996-4b1f0da6d5df None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:17.290783 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc2225-f5a1-43e9-805e-ffd35b5ab8dd None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:17.310102 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:17.328745 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f6a4a-c4d0-441b-bb92-7101858d984c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:17.329395 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f6a4a-c4d0-441b-bb92-7101858d984c None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:17.329677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f6a4a-c4d0-441b-bb92-7101858d984c None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:17.329951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f6a4a-c4d0-441b-bb92-7101858d984c None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:17.330744 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f6a4a-c4d0-441b-bb92-7101858d984c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:17.331048 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f6a4a-c4d0-441b-bb92-7101858d984c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:17.333236 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f6a4a-c4d0-441b-bb92-7101858d984c None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:17.345213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4882ba-16fd-4366-8122-3f2f66a1ae30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:17.345856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4882ba-16fd-4366-8122-3f2f66a1ae30 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:17.346145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4882ba-16fd-4366-8122-3f2f66a1ae30 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:17.346426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4882ba-16fd-4366-8122-3f2f66a1ae30 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:17.347204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4882ba-16fd-4366-8122-3f2f66a1ae30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:17.347512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4882ba-16fd-4366-8122-3f2f66a1ae30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:17.349245 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4882ba-16fd-4366-8122-3f2f66a1ae30 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:17.353333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 342/689] 10.222.0.74 () {56 vars in 1133 bytes} [Wed Jul 27 16:23:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:17.361878 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f6a4a-c4d0-441b-bb92-7101858d984c None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:17.379140 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 348/690] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:17.384990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4882ba-16fd-4366-8122-3f2f66a1ae30 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:17.404092 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:17.515292 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f6a4a-c4d0-441b-bb92-7101858d984c None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:17.548193 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4882ba-16fd-4366-8122-3f2f66a1ae30 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:17.556985 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545d3a4-40f2-480e-9d2d-0c89ce522528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:17.558175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545d3a4-40f2-480e-9d2d-0c89ce522528 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:17.558989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545d3a4-40f2-480e-9d2d-0c89ce522528 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:17.559431 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545d3a4-40f2-480e-9d2d-0c89ce522528 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:17.611148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:17.637624 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545d3a4-40f2-480e-9d2d-0c89ce522528 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:17.639350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545d3a4-40f2-480e-9d2d-0c89ce522528 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:17.646620 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba25c0-4ae6-4165-9046-6a730d1739c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:17.647387 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba25c0-4ae6-4165-9046-6a730d1739c3 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:17.647783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba25c0-4ae6-4165-9046-6a730d1739c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:17.648548 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba25c0-4ae6-4165-9046-6a730d1739c3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:17.666176 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba25c0-4ae6-4165-9046-6a730d1739c3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:17.667767 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba25c0-4ae6-4165-9046-6a730d1739c3 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:17.691572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-4cba25c0-4ae6-4165-9046-6a730d1739c3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 16:23:17.691572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:17.691572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:17.691572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:17.691572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:17.691572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:17.691572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:17.691572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:17.691572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:17.691572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:17.691572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:17.691572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:17.691572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:17.691572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:17.691572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:17.691572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:17.692777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:17.692777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:17.692777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:17.692777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:17.692777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:17.692777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:17.692777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:17.692777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:17.692777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:17.692777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:17.692777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:17.692777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:17.692777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:17.692777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:17.692777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:17.692777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:17.692777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:17.692777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:17.693971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:17.693971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:17.693971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:17.693971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:17.693971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:17.693971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:17.693971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:17.693971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:17.693971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 16:23:17.693971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:17.697710 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 349/691] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:23:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:23:17.699269 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba25c0-4ae6-4165-9046-6a730d1739c3 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:17.759150 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 343/692] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:23:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:23:17.768254 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545d3a4-40f2-480e-9d2d-0c89ce522528 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:17.857064 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:17.870708 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:17.887205 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545d3a4-40f2-480e-9d2d-0c89ce522528 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:17.926738 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78aa20-2d5c-40b4-b581-3078b15f62a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:17.928058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78aa20-2d5c-40b4-b581-3078b15f62a4 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:17.928500 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78aa20-2d5c-40b4-b581-3078b15f62a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:17.929097 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78aa20-2d5c-40b4-b581-3078b15f62a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:17.941211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78aa20-2d5c-40b4-b581-3078b15f62a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:17.943423 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78aa20-2d5c-40b4-b581-3078b15f62a4 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:17.957561 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba25c0-4ae6-4165-9046-6a730d1739c3 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:17.959993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-9d78aa20-2d5c-40b4-b581-3078b15f62a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:17.959993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:17.959993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:17.959993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:17.959993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:17.959993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:17.959993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:17.959993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:17.959993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:17.959993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:17.959993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:17.959993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:17.959993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:17.959993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:17.959993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:17.961039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:17.961039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:17.961039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:17.961039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:17.961039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:17.961039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:17.961039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:17.961039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:17.961039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:17.961039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:17.961039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:17.961039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:17.961039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:17.961039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:17.961039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:17.961039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:17.961039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:17.962154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:17.962154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:17.962154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:17.962154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:17.964917 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 344/693] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:17.970372 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78aa20-2d5c-40b4-b581-3078b15f62a4 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:18.038540 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addbbd2-619a-4014-a35e-5282b03aee5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:18.039169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addbbd2-619a-4014-a35e-5282b03aee5d None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:18.039452 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addbbd2-619a-4014-a35e-5282b03aee5d None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:18.039727 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addbbd2-619a-4014-a35e-5282b03aee5d None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:18.040639 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6addbbd2-619a-4014-a35e-5282b03aee5d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:18.040971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6addbbd2-619a-4014-a35e-5282b03aee5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:18.043090 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6addbbd2-619a-4014-a35e-5282b03aee5d None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:18.080384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 350/694] 10.222.0.74 () {56 vars in 1135 bytes} [Wed Jul 27 16:23:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:18.085507 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addbbd2-619a-4014-a35e-5282b03aee5d None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:18.093352 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78aa20-2d5c-40b4-b581-3078b15f62a4 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:18.132713 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:18.137081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e1f05-9c52-430f-a5e5-9242bfa715f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:18.137904 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e1f05-9c52-430f-a5e5-9242bfa715f7 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:18.138741 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e1f05-9c52-430f-a5e5-9242bfa715f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:18.139196 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e1f05-9c52-430f-a5e5-9242bfa715f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:18.140229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e1f05-9c52-430f-a5e5-9242bfa715f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:18.140813 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e1f05-9c52-430f-a5e5-9242bfa715f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:18.142783 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e1f05-9c52-430f-a5e5-9242bfa715f7 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:18.157551 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 345/695] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:18.162537 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e1f05-9c52-430f-a5e5-9242bfa715f7 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:18.283462 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e1f05-9c52-430f-a5e5-9242bfa715f7 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:18.324787 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62d3dc-c0c7-4825-b298-024a72490c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:18.326336 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62d3dc-c0c7-4825-b298-024a72490c57 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:18.326791 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62d3dc-c0c7-4825-b298-024a72490c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:18.327233 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62d3dc-c0c7-4825-b298-024a72490c57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:18.327702 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addbbd2-619a-4014-a35e-5282b03aee5d None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:18.336968 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62d3dc-c0c7-4825-b298-024a72490c57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:18.339898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62d3dc-c0c7-4825-b298-024a72490c57 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:18.360410 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-be62d3dc-c0c7-4825-b298-024a72490c57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:18.360410 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:18.360410 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:18.360410 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:18.360410 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:18.360410 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:18.360410 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:18.360410 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:18.360410 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:18.360410 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:18.360410 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:18.360410 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:18.360410 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:18.360410 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:18.360410 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:18.361646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:18.361646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:18.361646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:18.361646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:18.361646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:18.361646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:18.361646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:18.361646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:18.361646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:18.361646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:18.361646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:18.361646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:18.361646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:18.361646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:18.361646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:18.361646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:18.361646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:18.362736 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:18.362736 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:18.362736 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:18.362736 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:18.362736 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:18.362736 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:18.362736 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:18.362736 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:18.365082 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:18.365525 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 346/696] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:18.369916 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32241945-2294-42c6-808a-435fd1fccac2 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:18.371614 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32241945-2294-42c6-808a-435fd1fccac2 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:18.372636 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32241945-2294-42c6-808a-435fd1fccac2 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:18.374430 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 347/697] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:18.375345 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32241945-2294-42c6-808a-435fd1fccac2 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:18.380584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0d221-a1ed-4807-8f7f-f3a7fc409f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:18.381586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0d221-a1ed-4807-8f7f-f3a7fc409f57 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:18.382071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0d221-a1ed-4807-8f7f-f3a7fc409f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:18.382629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0d221-a1ed-4807-8f7f-f3a7fc409f57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:18.432067 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:18.461249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f0d221-a1ed-4807-8f7f-f3a7fc409f57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:18.463187 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f0d221-a1ed-4807-8f7f-f3a7fc409f57 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:18.488928 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32241945-2294-42c6-808a-435fd1fccac2 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:18.525246 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1500e-0d38-44fa-9a07-07bfcd960c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:18.525862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1500e-0d38-44fa-9a07-07bfcd960c8f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:18.526145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1500e-0d38-44fa-9a07-07bfcd960c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:18.526423 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1500e-0d38-44fa-9a07-07bfcd960c8f None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:18.527207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d1500e-0d38-44fa-9a07-07bfcd960c8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:18.527517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d1500e-0d38-44fa-9a07-07bfcd960c8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:18.528967 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d1500e-0d38-44fa-9a07-07bfcd960c8f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:18.543383 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 348/698] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:18.544688 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dbc33-738c-4fa7-b959-cf56f56f0552 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:18.544999 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dbc33-738c-4fa7-b959-cf56f56f0552 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:18.545279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dbc33-738c-4fa7-b959-cf56f56f0552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:18.563291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-1d8dbc33-738c-4fa7-b959-cf56f56f0552 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:18.564487 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-1d8dbc33-738c-4fa7-b959-cf56f56f0552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:18.579863 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 351/699] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:23:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 16:23:18.581376 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0d221-a1ed-4807-8f7f-f3a7fc409f57 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:18.610976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:18.645164 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 349/700] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:23:18.646048 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dbc33-738c-4fa7-b959-cf56f56f0552 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:18.696237 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0d221-a1ed-4807-8f7f-f3a7fc409f57 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:18.708766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:18.737671 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71ce26-d073-456c-b8c8-18929bf8c0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:18.738672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71ce26-d073-456c-b8c8-18929bf8c0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:18.739115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71ce26-d073-456c-b8c8-18929bf8c0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:18.739557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71ce26-d073-456c-b8c8-18929bf8c0f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:18.740498 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71ce26-d073-456c-b8c8-18929bf8c0f7 None admin] RBAC: Adding query filter params (role.id=ca37e43082fa40c6a88b12680a0ca94b, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=56dfafd15e284b7986ba07dd3819bf65) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:18.740993 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71ce26-d073-456c-b8c8-18929bf8c0f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:18.742776 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71ce26-d073-456c-b8c8-18929bf8c0f7 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:18.749784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 352/701] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:23:18 2022] GET /identity/v3/role_assignments?user.id=56dfafd15e284b7986ba07dd3819bf65&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=ca37e43082fa40c6a88b12680a0ca94b => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:18.750870 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71ce26-d073-456c-b8c8-18929bf8c0f7 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:18.777329 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dbc33-738c-4fa7-b959-cf56f56f0552 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:18.818195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fb8aa-a58f-4abd-bc44-876bfd384fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:18.818889 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fb8aa-a58f-4abd-bc44-876bfd384fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:18.819093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fb8aa-a58f-4abd-bc44-876bfd384fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:18.819483 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fb8aa-a58f-4abd-bc44-876bfd384fd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:18.832195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12fb8aa-a58f-4abd-bc44-876bfd384fd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:18.833881 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12fb8aa-a58f-4abd-bc44-876bfd384fd2 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:18.850280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-a12fb8aa-a58f-4abd-bc44-876bfd384fd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:18.850280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:18.850280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:18.850280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:18.850280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:18.850280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:18.850280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:18.850280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:18.850280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:18.850280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:18.850280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:18.850280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:18.850280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:18.850280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:18.850280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:18.851509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:18.851509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:18.851509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:18.851509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:18.851509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:18.851509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:18.851509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:18.851509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:18.851509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:18.851509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:18.851509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:18.851509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:18.851509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:18.851509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:18.851509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:18.851509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:18.851509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:18.852739 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:18.852739 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:18.852739 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:18.852739 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:18.852739 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 350/702] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:18.857340 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fb8aa-a58f-4abd-bc44-876bfd384fd2 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:18.888861 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:18.911159 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71ce26-d073-456c-b8c8-18929bf8c0f7 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:18.972800 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1090de7-2f37-4436-a560-182699235eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:18.973931 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1090de7-2f37-4436-a560-182699235eba None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:18.975294 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1090de7-2f37-4436-a560-182699235eba None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:18.978421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1090de7-2f37-4436-a560-182699235eba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:18.990119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1090de7-2f37-4436-a560-182699235eba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:18.991609 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1090de7-2f37-4436-a560-182699235eba None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:19.008617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-b1090de7-2f37-4436-a560-182699235eba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:19.008617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:19.008617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:19.008617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:19.008617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:19.008617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:19.008617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:19.008617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:19.008617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:19.008617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:19.008617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:19.008617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:19.008617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:19.008617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:19.008617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:19.009760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:19.009760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:19.009760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:19.009760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:19.009760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:19.009760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:19.009760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:19.009760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:19.009760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:19.009760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:19.009760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:19.009760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:19.009760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:19.009760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:19.009760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:19.009760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:19.009760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:19.010639 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:19.010639 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:19.010639 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:19.010639 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:19.018640 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 353/703] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:19.029936 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1090de7-2f37-4436-a560-182699235eba None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:19.034960 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:19.051796 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fb8aa-a58f-4abd-bc44-876bfd384fd2 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:19.129658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f45923-a196-4af8-bdde-b7ed383f995a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:19.131807 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f45923-a196-4af8-bdde-b7ed383f995a None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:19.132646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f45923-a196-4af8-bdde-b7ed383f995a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:19.134948 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f45923-a196-4af8-bdde-b7ed383f995a None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:19.136916 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f45923-a196-4af8-bdde-b7ed383f995a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:19.138269 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f45923-a196-4af8-bdde-b7ed383f995a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:19.140535 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f45923-a196-4af8-bdde-b7ed383f995a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:19.168331 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 351/704] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:19.178010 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f45923-a196-4af8-bdde-b7ed383f995a None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:19.230120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1090de7-2f37-4436-a560-182699235eba None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:19.283015 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43633e78-42e9-4a55-b651-8d2b10194bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:19.283640 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43633e78-42e9-4a55-b651-8d2b10194bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:19.283924 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43633e78-42e9-4a55-b651-8d2b10194bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:19.284199 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43633e78-42e9-4a55-b651-8d2b10194bb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:19.284992 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43633e78-42e9-4a55-b651-8d2b10194bb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:19.285298 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43633e78-42e9-4a55-b651-8d2b10194bb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:19.286676 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43633e78-42e9-4a55-b651-8d2b10194bb0 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:19.310383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 354/705] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:19.312749 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43633e78-42e9-4a55-b651-8d2b10194bb0 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:19.316846 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:19.383298 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f45923-a196-4af8-bdde-b7ed383f995a None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:19.453562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe2700-b28e-416f-8d87-3e58176bf887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:19.454227 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe2700-b28e-416f-8d87-3e58176bf887 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:19.454787 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe2700-b28e-416f-8d87-3e58176bf887 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:19.455070 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe2700-b28e-416f-8d87-3e58176bf887 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:19.465255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fe2700-b28e-416f-8d87-3e58176bf887 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:19.466136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43633e78-42e9-4a55-b651-8d2b10194bb0 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:19.466717 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fe2700-b28e-416f-8d87-3e58176bf887 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:19.481026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-24fe2700-b28e-416f-8d87-3e58176bf887 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:19.481026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:19.481026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:19.481026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:19.481026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:19.481026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:19.481026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:19.481026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:19.481026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:19.481026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:19.481026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:19.481026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:19.481026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:19.481026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:19.481026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:19.481922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:19.481922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:19.481922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:19.481922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:19.481922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:19.481922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:19.481922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:19.481922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:19.481922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:19.481922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:19.481922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:19.481922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:19.481922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:19.481922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:19.481922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:19.481922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:19.481922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:19.483685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:19.483685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:19.483685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:19.483685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:19.483685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:19.483685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:19.483685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:19.483685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:19.483685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 352/706] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:19.483685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde20153-59df-402c-859c-46d422b00e15 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:19.483685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde20153-59df-402c-859c-46d422b00e15 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:19.485189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde20153-59df-402c-859c-46d422b00e15 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:19.489223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 353/707] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:19.490396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d9b5a-8e48-425c-9ebf-9e075de1d367 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:19.490678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d9b5a-8e48-425c-9ebf-9e075de1d367 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:19.490951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d9b5a-8e48-425c-9ebf-9e075de1d367 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:19.492303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 354/708] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:19.496932 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d9b5a-8e48-425c-9ebf-9e075de1d367 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:19.507180 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:19.512128 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3e8f3-a76a-44ab-8306-20919f3b25e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:19.512758 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3e8f3-a76a-44ab-8306-20919f3b25e7 None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:19.513232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3e8f3-a76a-44ab-8306-20919f3b25e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:19.513430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3e8f3-a76a-44ab-8306-20919f3b25e7 None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:19.514089 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3e8f3-a76a-44ab-8306-20919f3b25e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:19.515561 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3e8f3-a76a-44ab-8306-20919f3b25e7 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:19.539404 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 355/709] 10.222.0.74 () {60 vars in 1132 bytes} [Wed Jul 27 16:23:19 2022] POST /identity/v3/users => generated 254 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:23:19.540708 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f802a-3bc5-4480-8a1a-fd62ed373351 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:19.541142 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f802a-3bc5-4480-8a1a-fd62ed373351 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:19.541427 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f802a-3bc5-4480-8a1a-fd62ed373351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:19.562706 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-1b1f802a-3bc5-4480-8a1a-fd62ed373351 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:19.564010 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-1b1f802a-3bc5-4480-8a1a-fd62ed373351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:19.623978 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d9b5a-8e48-425c-9ebf-9e075de1d367 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:19.633525 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:19.656845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 356/710] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:23:19.659423 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44dfb18-24c1-40bb-a33b-227f74e90726 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:19.660008 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44dfb18-24c1-40bb-a33b-227f74e90726 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:19.660596 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44dfb18-24c1-40bb-a33b-227f74e90726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:19.662361 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecff65a-5dee-4b32-86c1-781854fb1ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:19.663085 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecff65a-5dee-4b32-86c1-781854fb1ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:19.663373 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecff65a-5dee-4b32-86c1-781854fb1ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:19.663698 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecff65a-5dee-4b32-86c1-781854fb1ad5 None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:19.664509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecff65a-5dee-4b32-86c1-781854fb1ad5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:19.664812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecff65a-5dee-4b32-86c1-781854fb1ad5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:19.666276 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecff65a-5dee-4b32-86c1-781854fb1ad5 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:19.684207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-b44dfb18-24c1-40bb-a33b-227f74e90726 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:19.684677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 355/711] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:19.685956 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecff65a-5dee-4b32-86c1-781854fb1ad5 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:19.686404 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b44dfb18-24c1-40bb-a33b-227f74e90726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:19.772670 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:19.779822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 357/712] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:23:19.784960 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44dfb18-24c1-40bb-a33b-227f74e90726 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:19.812362 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecff65a-5dee-4b32-86c1-781854fb1ad5 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:19.861358 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b177761-5d0e-400c-8274-7186839e0718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:19.861987 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b177761-5d0e-400c-8274-7186839e0718 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:19.862268 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b177761-5d0e-400c-8274-7186839e0718 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:19.862542 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b177761-5d0e-400c-8274-7186839e0718 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:19.863329 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b177761-5d0e-400c-8274-7186839e0718 None admin] RBAC: Adding query filter params (role.id=2aa0cdfed849496db8f3692c9569d7ee, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=bb45ded9f75e4fa2973b564f83bdfaf2) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:19.863640 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b177761-5d0e-400c-8274-7186839e0718 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:19.865124 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b177761-5d0e-400c-8274-7186839e0718 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:19.872211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 356/713] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:23:19 2022] GET /identity/v3/role_assignments?user.id=bb45ded9f75e4fa2973b564f83bdfaf2&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=2aa0cdfed849496db8f3692c9569d7ee => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:19.872639 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b177761-5d0e-400c-8274-7186839e0718 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:19.917680 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44dfb18-24c1-40bb-a33b-227f74e90726 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:19.941926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:19.982686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ded23-20f4-4541-9884-d82208d0222b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:19.983364 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ded23-20f4-4541-9884-d82208d0222b None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:19.983640 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ded23-20f4-4541-9884-d82208d0222b None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:19.983906 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ded23-20f4-4541-9884-d82208d0222b None admin] PATH_INFO: `/v3/services/network` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:19.984706 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03ded23-20f4-4541-9884-d82208d0222b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:19.989962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03ded23-20f4-4541-9884-d82208d0222b None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:20.009750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-c03ded23-20f4-4541-9884-d82208d0222b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 16:23:20.009750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:20.009750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:23:20.009750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:23:20.009750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:23:20.009750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:23:20.009750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:23:20.009750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:20.009750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 16:23:20.009750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:20.009750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:20.009750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:20.009750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:20.009750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:20.009750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:20.009750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:20.009750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:20.011563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:20.011563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:20.011563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:20.011563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:20.011563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:20.011563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:20.011563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:23:20.011563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:23:20.011563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:23:20.011563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:23:20.011563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:20.011563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:20.011563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:20.011563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:20.011563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:20.011563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:20.012480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:20.012480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:20.012480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:20.012480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:20.012480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:20.012480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:20.012480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:20.012480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:20.012480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:20.012480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:20.012480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:23:20.012480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:20.012480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 16:23:20.012480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:20.015062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b177761-5d0e-400c-8274-7186839e0718 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:20.017626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 358/714] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:19 2022] GET /identity/v3/services/network => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:20.023428 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ded23-20f4-4541-9884-d82208d0222b None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:20.064059 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc691631-ed14-4ee2-b5ff-0af7688eb2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:20.065353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc691631-ed14-4ee2-b5ff-0af7688eb2f1 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:20.065902 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc691631-ed14-4ee2-b5ff-0af7688eb2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:20.066385 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc691631-ed14-4ee2-b5ff-0af7688eb2f1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:20.067695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc691631-ed14-4ee2-b5ff-0af7688eb2f1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:20.069880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc691631-ed14-4ee2-b5ff-0af7688eb2f1 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:20.085806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-bc691631-ed14-4ee2-b5ff-0af7688eb2f1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 16:23:20.085806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:20.085806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:23:20.085806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:23:20.085806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:23:20.085806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:23:20.085806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:23:20.085806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:20.085806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 16:23:20.085806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:20.085806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:20.085806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:20.085806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:20.085806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:20.085806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:20.085806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:20.086846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:20.086846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:20.086846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:20.086846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:20.086846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:20.086846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:20.086846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:20.086846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:23:20.086846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:23:20.086846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:23:20.086846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:23:20.086846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:20.086846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:20.086846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:20.086846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:20.086846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:20.087709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:20.087709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:20.087709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:20.087709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:20.087709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:20.087709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:20.087709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:20.087709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:20.087709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:20.087709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:20.087709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:20.087709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:23:20.087709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:20.087709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 16:23:20.087709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:20.095579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 357/715] 10.222.0.74 () {56 vars in 1129 bytes} [Wed Jul 27 16:23:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:20.095579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:20.097932 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc691631-ed14-4ee2-b5ff-0af7688eb2f1 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:20.179199 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ded23-20f4-4541-9884-d82208d0222b None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:20.220904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46e0ba-caa4-46de-9ade-a96566080be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:20.221565 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46e0ba-caa4-46de-9ade-a96566080be1 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:20.221854 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46e0ba-caa4-46de-9ade-a96566080be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:20.222181 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46e0ba-caa4-46de-9ade-a96566080be1 None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:20.222967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46e0ba-caa4-46de-9ade-a96566080be1 None admin] RBAC: Adding query filter params (name=network) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:20.223285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46e0ba-caa4-46de-9ade-a96566080be1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:20.224674 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46e0ba-caa4-46de-9ade-a96566080be1 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:20.231424 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 359/716] 10.222.0.74 () {56 vars in 1112 bytes} [Wed Jul 27 16:23:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:20.236828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc691631-ed14-4ee2-b5ff-0af7688eb2f1 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:20.237099 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46e0ba-caa4-46de-9ade-a96566080be1 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:20.291553 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:20.315465 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e2cfa-b31d-48d7-b64c-342219d39cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:20.316098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e2cfa-b31d-48d7-b64c-342219d39cdc None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:20.316383 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e2cfa-b31d-48d7-b64c-342219d39cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:20.316653 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e2cfa-b31d-48d7-b64c-342219d39cdc None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:20.317600 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e2cfa-b31d-48d7-b64c-342219d39cdc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:20.317917 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e2cfa-b31d-48d7-b64c-342219d39cdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:20.320199 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e2cfa-b31d-48d7-b64c-342219d39cdc None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:20.332078 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 358/717] 10.222.0.74 () {56 vars in 1124 bytes} [Wed Jul 27 16:23:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:20.333654 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e2cfa-b31d-48d7-b64c-342219d39cdc None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:20.357671 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46e0ba-caa4-46de-9ade-a96566080be1 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:20.398313 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2919d-8343-4185-807f-979893f108cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:20.399403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2919d-8343-4185-807f-979893f108cc None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:20.399893 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2919d-8343-4185-807f-979893f108cc None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:20.400378 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2919d-8343-4185-807f-979893f108cc None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:20.401303 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2919d-8343-4185-807f-979893f108cc None admin] RBAC: Adding query filter params (type=network) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:20.402446 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2919d-8343-4185-807f-979893f108cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:20.403877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2919d-8343-4185-807f-979893f108cc None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:20.410051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 360/718] 10.222.0.74 () {56 vars in 1112 bytes} [Wed Jul 27 16:23:20 2022] GET /identity/v3/services?type=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:20.428657 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:20.512345 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e2cfa-b31d-48d7-b64c-342219d39cdc None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:20.557258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6214c7-d021-4d6d-aef7-c180dc44ba49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:20.558792 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6214c7-d021-4d6d-aef7-c180dc44ba49 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:20.559256 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6214c7-d021-4d6d-aef7-c180dc44ba49 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:20.559734 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6214c7-d021-4d6d-aef7-c180dc44ba49 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:20.560593 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6214c7-d021-4d6d-aef7-c180dc44ba49 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:20.561127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6214c7-d021-4d6d-aef7-c180dc44ba49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:20.562676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6214c7-d021-4d6d-aef7-c180dc44ba49 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:20.574658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 359/719] 10.222.0.74 () {56 vars in 1124 bytes} [Wed Jul 27 16:23:20 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:21.045803 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37490697-8d3d-4fcb-89dd-5b20b580d27d None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:21.046324 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37490697-8d3d-4fcb-89dd-5b20b580d27d None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:21.046436 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37490697-8d3d-4fcb-89dd-5b20b580d27d None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:21.047963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 361/720] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:21.053060 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a750e-96b0-45fc-accb-bf852c738d47 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:21.053452 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a750e-96b0-45fc-accb-bf852c738d47 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:21.053620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a750e-96b0-45fc-accb-bf852c738d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:21.076022 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-741a750e-96b0-45fc-accb-bf852c738d47 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:21.077260 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-741a750e-96b0-45fc-accb-bf852c738d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:21.159647 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 360/721] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:23:21.199143 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37490697-8d3d-4fcb-89dd-5b20b580d27d None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:21.327006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37490697-8d3d-4fcb-89dd-5b20b580d27d None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:21.337864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:21.369778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f16f69-a523-40ce-a524-2d29cbe5971d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:21.370464 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f16f69-a523-40ce-a524-2d29cbe5971d None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:21.370750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f16f69-a523-40ce-a524-2d29cbe5971d None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:21.371029 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f16f69-a523-40ce-a524-2d29cbe5971d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:21.385166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f16f69-a523-40ce-a524-2d29cbe5971d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:21.391241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-b7f16f69-a523-40ce-a524-2d29cbe5971d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:23:21.391241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:21.391241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:21.391241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:21.391241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:21.391241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:21.391241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:21.391241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:21.391241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:21.391241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:21.391241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:21.391241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:21.391241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:21.391241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:21.391241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:21.391241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:21.392172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:21.392172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:21.392172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:21.392172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:21.392172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:21.392172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:21.392172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:21.392172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:21.392172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:21.392172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:21.392172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:21.392172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:21.392172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:21.392172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:21.392172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:21.392172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:21.393059 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:21.393059 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:21.393059 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:21.393059 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:21.393059 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:21.393059 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:21.393059 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:21.393059 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:21.393059 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:21.393059 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:21.393059 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:23:21.393059 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:21.393059 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 362/722] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:23:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:21.400129 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a750e-96b0-45fc-accb-bf852c738d47 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:21.495853 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:21.548048 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a750e-96b0-45fc-accb-bf852c738d47 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:21.582463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f14194-6405-4068-b86d-15703eadaffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:21.583006 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f14194-6405-4068-b86d-15703eadaffb None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:21.583288 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f14194-6405-4068-b86d-15703eadaffb None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:21.583563 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f14194-6405-4068-b86d-15703eadaffb None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:21.584284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f14194-6405-4068-b86d-15703eadaffb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:21.584595 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f14194-6405-4068-b86d-15703eadaffb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:21.586012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f14194-6405-4068-b86d-15703eadaffb None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:21.594919 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 361/723] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:23:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:21.601862 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f16f69-a523-40ce-a524-2d29cbe5971d None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:21.742132 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f16f69-a523-40ce-a524-2d29cbe5971d None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:21.787775 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee675bba-6cfa-4a38-9591-da78733b6dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:21.788449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee675bba-6cfa-4a38-9591-da78733b6dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:21.788745 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee675bba-6cfa-4a38-9591-da78733b6dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:21.789051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee675bba-6cfa-4a38-9591-da78733b6dc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:21.801166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee675bba-6cfa-4a38-9591-da78733b6dc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:21.802772 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee675bba-6cfa-4a38-9591-da78733b6dc3 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:21.818686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-ee675bba-6cfa-4a38-9591-da78733b6dc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:21.818686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:21.818686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:21.818686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:21.818686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:21.818686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:21.818686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:21.818686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:21.818686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:21.818686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:21.818686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:21.818686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:21.818686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:21.818686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:21.818686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:21.819652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:21.819652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:21.819652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:21.819652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:21.819652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:21.819652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:21.819652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:21.819652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:21.819652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:21.819652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:21.819652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:21.819652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:21.819652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:21.819652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:21.819652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:21.819652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:21.819652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:21.820526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:21.820526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:21.820526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:21.820526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:21.820526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 363/724] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:21.827029 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f14194-6405-4068-b86d-15703eadaffb None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:21.992722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f14194-6405-4068-b86d-15703eadaffb None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:22.018086 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:22.038770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca0f5b-5961-4ff8-bf74-f1762caec6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:22.039392 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca0f5b-5961-4ff8-bf74-f1762caec6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:22.039673 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca0f5b-5961-4ff8-bf74-f1762caec6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:22.039945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca0f5b-5961-4ff8-bf74-f1762caec6c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:22.040634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca0f5b-5961-4ff8-bf74-f1762caec6c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:22.040963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca0f5b-5961-4ff8-bf74-f1762caec6c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:22.042370 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca0f5b-5961-4ff8-bf74-f1762caec6c9 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:22.063393 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 362/725] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:22.069484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee675bba-6cfa-4a38-9591-da78733b6dc3 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:22.128169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:22.160462 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6763b42-6498-4e19-82ba-ce78abf3f0d2 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:22.160805 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6763b42-6498-4e19-82ba-ce78abf3f0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:22.161058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6763b42-6498-4e19-82ba-ce78abf3f0d2 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:22.165399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 363/726] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:23:22.168354 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69eeba5-82d6-4b49-8d01-7cd7357a6bcb None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:22.168452 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69eeba5-82d6-4b49-8d01-7cd7357a6bcb None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:22.168678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69eeba5-82d6-4b49-8d01-7cd7357a6bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:22.197071 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-b69eeba5-82d6-4b49-8d01-7cd7357a6bcb None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:22.197071 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b69eeba5-82d6-4b49-8d01-7cd7357a6bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:22.294812 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee675bba-6cfa-4a38-9591-da78733b6dc3 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:22.335081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 364/727] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 16:23:22.335425 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9c52f-c30f-45e3-b142-61219e1bc1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:22.336144 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9c52f-c30f-45e3-b142-61219e1bc1b8 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:22.336432 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9c52f-c30f-45e3-b142-61219e1bc1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:22.336711 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9c52f-c30f-45e3-b142-61219e1bc1b8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:22.357591 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69eeba5-82d6-4b49-8d01-7cd7357a6bcb None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:22.359359 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9c52f-c30f-45e3-b142-61219e1bc1b8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:22.361054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9c52f-c30f-45e3-b142-61219e1bc1b8 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:22.373557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-a9e9c52f-c30f-45e3-b142-61219e1bc1b8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 16:23:22.373557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:22.373557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:22.373557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:22.373557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:22.373557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:22.373557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:22.373557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:22.373557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:22.373557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:22.373557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:22.373557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:22.373557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:22.373557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:22.373557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:22.373557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:22.374523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:22.374523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:22.374523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:22.374523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:22.374523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:22.374523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:22.374523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:22.374523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:22.374523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:22.374523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:22.374523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:22.374523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:22.374523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:22.374523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:22.374523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:22.374523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:22.374523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:22.374523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:22.375451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:22.375451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:22.375451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:22.375451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:22.375451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:22.375451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:22.375451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:22.375451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:22.375451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 16:23:22.375451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:22.377992 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 364/728] 10.222.0.74 () {56 vars in 1093 bytes} [Wed Jul 27 16:23:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:22.384981 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9c52f-c30f-45e3-b142-61219e1bc1b8 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:22.525684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69eeba5-82d6-4b49-8d01-7cd7357a6bcb None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:22.543366 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:22.543975 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:22.577515 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9c52f-c30f-45e3-b142-61219e1bc1b8 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:22.591515 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa696189-5dca-49bc-9d53-ad76c72483e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:22.592173 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa696189-5dca-49bc-9d53-ad76c72483e0 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:22.592457 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa696189-5dca-49bc-9d53-ad76c72483e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:22.592731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa696189-5dca-49bc-9d53-ad76c72483e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:22.609696 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa696189-5dca-49bc-9d53-ad76c72483e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:22.614675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-fa696189-5dca-49bc-9d53-ad76c72483e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:22.614675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:22.614675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:22.614675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:22.614675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:22.614675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:22.614675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:22.614675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:22.614675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:22.614675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:22.614675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:22.614675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:22.614675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:22.614675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:22.614675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:22.614675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:22.615562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:22.615562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:22.615562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:22.615562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:22.615562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:22.615562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:22.615562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:22.615562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:22.615562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:22.615562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:22.615562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:22.615562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:22.615562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:22.615562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:22.615562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:22.615562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:22.616402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:22.616402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:22.616402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:22.616402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:22.616402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:22.616402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:22.616402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:22.616402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:22.616402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:22.616402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:22.616402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:22.616402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:22.620773 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 365/729] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:23:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:23:22.625818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa696189-5dca-49bc-9d53-ad76c72483e0 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:22.645893 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759f7e6-6323-4e0b-ad1a-625af63ee68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:22.646894 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759f7e6-6323-4e0b-ad1a-625af63ee68e None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:22.647379 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759f7e6-6323-4e0b-ad1a-625af63ee68e None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:22.647875 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759f7e6-6323-4e0b-ad1a-625af63ee68e None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:22.649059 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759f7e6-6323-4e0b-ad1a-625af63ee68e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:22.649599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759f7e6-6323-4e0b-ad1a-625af63ee68e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:22.651332 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759f7e6-6323-4e0b-ad1a-625af63ee68e None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:22.678750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 365/730] 10.222.0.74 () {56 vars in 1133 bytes} [Wed Jul 27 16:23:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:22.685841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759f7e6-6323-4e0b-ad1a-625af63ee68e None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:22.793280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa696189-5dca-49bc-9d53-ad76c72483e0 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:22.831604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc387d69-db5e-41c7-8679-e91c3a6adf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:22.832160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc387d69-db5e-41c7-8679-e91c3a6adf2f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:22.832449 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc387d69-db5e-41c7-8679-e91c3a6adf2f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:22.832737 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc387d69-db5e-41c7-8679-e91c3a6adf2f None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:22.833647 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc387d69-db5e-41c7-8679-e91c3a6adf2f None admin] RBAC: Adding query filter params (name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:22.833962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc387d69-db5e-41c7-8679-e91c3a6adf2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:22.835359 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc387d69-db5e-41c7-8679-e91c3a6adf2f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:22.845370 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 366/731] 10.222.0.74 () {56 vars in 1103 bytes} [Wed Jul 27 16:23:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:22.846561 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ea478-865f-4e43-ad8e-7d1c52c2006c None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:22.846859 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ea478-865f-4e43-ad8e-7d1c52c2006c None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:22.847134 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ea478-865f-4e43-ad8e-7d1c52c2006c None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:22.848606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 367/732] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:22.855761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ea478-865f-4e43-ad8e-7d1c52c2006c None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:22.876533 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759f7e6-6323-4e0b-ad1a-625af63ee68e None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:22.889991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:22.919492 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521261cc-27e8-49e3-b3cd-a531ebc78674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:22.920160 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521261cc-27e8-49e3-b3cd-a531ebc78674 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:22.920443 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521261cc-27e8-49e3-b3cd-a531ebc78674 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:22.920721 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521261cc-27e8-49e3-b3cd-a531ebc78674 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:22.929593 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:22.932030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521261cc-27e8-49e3-b3cd-a531ebc78674 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:22.933595 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521261cc-27e8-49e3-b3cd-a531ebc78674 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:22.950030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-521261cc-27e8-49e3-b3cd-a531ebc78674 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:22.950030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:22.950030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:22.950030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:22.950030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:22.950030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:22.950030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:22.950030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:22.950030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:22.950030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:22.950030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:22.950030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:22.950030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:22.950030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:22.950030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:22.951198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:22.951198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:22.951198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:22.951198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:22.951198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:22.951198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:22.951198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:22.951198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:22.951198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:22.951198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:22.951198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:22.951198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:22.951198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:22.951198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:22.951198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:22.951198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:22.951198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:22.952086 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:22.952086 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:22.952086 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:22.952086 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:22.952086 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 366/733] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:22.953090 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57c7b7-44a3-4e92-b8f2-08b20c80cfd9 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:22.953380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57c7b7-44a3-4e92-b8f2-08b20c80cfd9 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:22.953656 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57c7b7-44a3-4e92-b8f2-08b20c80cfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:22.972505 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-3b57c7b7-44a3-4e92-b8f2-08b20c80cfd9 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:22.973697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-3b57c7b7-44a3-4e92-b8f2-08b20c80cfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:22.981853 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ea478-865f-4e43-ad8e-7d1c52c2006c None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:23.017774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d4577-8921-443f-9e89-cab9c5fff07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:23.018778 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d4577-8921-443f-9e89-cab9c5fff07e None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:23.019265 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d4577-8921-443f-9e89-cab9c5fff07e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:23.020000 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d4577-8921-443f-9e89-cab9c5fff07e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:23.033260 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d4577-8921-443f-9e89-cab9c5fff07e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:23.035022 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d4577-8921-443f-9e89-cab9c5fff07e None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:23.052918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 367/734] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:23:23.052918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b57c7b7-44a3-4e92-b8f2-08b20c80cfd9 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:23.059629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-361d4577-8921-443f-9e89-cab9c5fff07e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:23.059629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:23.059629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:23.059629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:23.059629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:23.059629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:23.059629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:23.059629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:23.059629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:23.059629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:23.059629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:23.059629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:23.059629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:23.059629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:23.059629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:23.060509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:23.060509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:23.060509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:23.060509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:23.060509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:23.060509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:23.060509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:23.060509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:23.060509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:23.060509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:23.060509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:23.060509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:23.060509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:23.060509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:23.060509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:23.060509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:23.060509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:23.061419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:23.061419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:23.061419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:23.061419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:23.069394 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 368/735] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:23.069394 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe596f-2a73-4987-81c4-ff30d3d2b407 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:23.069394 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe596f-2a73-4987-81c4-ff30d3d2b407 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:23.069394 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe596f-2a73-4987-81c4-ff30d3d2b407 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:23.070125 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 369/736] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:23.072748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe596f-2a73-4987-81c4-ff30d3d2b407 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:23.164382 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:23.168632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b57c7b7-44a3-4e92-b8f2-08b20c80cfd9 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:23.185195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe596f-2a73-4987-81c4-ff30d3d2b407 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:23.191436 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:23.203630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5599bd-e43e-4a3b-b13e-fe2b7226a2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:23.204246 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5599bd-e43e-4a3b-b13e-fe2b7226a2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:23.204528 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5599bd-e43e-4a3b-b13e-fe2b7226a2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:23.204804 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5599bd-e43e-4a3b-b13e-fe2b7226a2a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:23.205522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5599bd-e43e-4a3b-b13e-fe2b7226a2a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:23.205823 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5599bd-e43e-4a3b-b13e-fe2b7226a2a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:23.207303 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5599bd-e43e-4a3b-b13e-fe2b7226a2a1 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:23.218391 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dcbf63-19e6-40ba-aa03-c2ee73ef50aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:23.218619 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 368/737] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:23.219028 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcbf63-19e6-40ba-aa03-c2ee73ef50aa None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:23.219317 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcbf63-19e6-40ba-aa03-c2ee73ef50aa None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:23.219674 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcbf63-19e6-40ba-aa03-c2ee73ef50aa None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:23.219780 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb61cc3-2098-4ea5-a4e1-0a1730dfc034 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:23.219985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb61cc3-2098-4ea5-a4e1-0a1730dfc034 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:23.220264 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb61cc3-2098-4ea5-a4e1-0a1730dfc034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:23.220369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dcbf63-19e6-40ba-aa03-c2ee73ef50aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:23.220669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dcbf63-19e6-40ba-aa03-c2ee73ef50aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:23.222069 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dcbf63-19e6-40ba-aa03-c2ee73ef50aa None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:23.234529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 370/738] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:23.235326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dcbf63-19e6-40ba-aa03-c2ee73ef50aa None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:23.238800 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-2eb61cc3-2098-4ea5-a4e1-0a1730dfc034 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:23.239961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-2eb61cc3-2098-4ea5-a4e1-0a1730dfc034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:23.305758 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 369/739] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:23:23.306988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5f57e-6978-448d-9a4a-2f931998408c None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:23.307324 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5f57e-6978-448d-9a4a-2f931998408c None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:23.307599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5f57e-6978-448d-9a4a-2f931998408c None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:23.313248 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 370/740] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:23.313248 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5f57e-6978-448d-9a4a-2f931998408c None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:23.336645 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dcbf63-19e6-40ba-aa03-c2ee73ef50aa None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:23.371390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714decab-c681-4c8d-b201-605568aa69f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:23.372025 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714decab-c681-4c8d-b201-605568aa69f2 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:23.372303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714decab-c681-4c8d-b201-605568aa69f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:23.372584 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714decab-c681-4c8d-b201-605568aa69f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:23.382663 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714decab-c681-4c8d-b201-605568aa69f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:23.387550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-714decab-c681-4c8d-b201-605568aa69f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:23:23.387550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:23.387550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:23.387550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:23.387550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:23.387550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:23.387550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:23.387550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:23.387550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:23.387550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:23.387550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:23.387550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:23.387550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:23.387550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:23.387550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:23.387550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:23.388728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:23.388728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:23.388728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:23.388728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:23.388728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:23.388728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:23.388728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:23.388728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:23.388728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:23.388728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:23.388728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:23.388728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:23.388728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:23.388728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:23.388728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:23.388728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:23.389665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:23.389665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:23.389665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:23.389665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:23.389665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:23.389665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:23.389665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:23.389665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:23.389665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:23.389665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:23.389665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:23:23.389665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:23.389665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 371/741] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:23:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:23.390351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714decab-c681-4c8d-b201-605568aa69f2 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:23.393251 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:23.419500 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5f57e-6978-448d-9a4a-2f931998408c None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:23.432523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:23.453910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa2078-e218-4551-90c9-18a1bf66b71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:23.454550 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa2078-e218-4551-90c9-18a1bf66b71f None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:23.454832 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa2078-e218-4551-90c9-18a1bf66b71f None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:23.455106 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa2078-e218-4551-90c9-18a1bf66b71f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:23.461673 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa2078-e218-4551-90c9-18a1bf66b71f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:23.463120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa2078-e218-4551-90c9-18a1bf66b71f None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:23.474942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-3bfa2078-e218-4551-90c9-18a1bf66b71f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:23.474942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:23.474942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:23.474942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:23.474942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:23.474942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:23.474942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:23.474942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:23.474942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:23.474942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:23.474942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:23.474942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:23.474942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:23.474942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:23.474942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:23.475783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:23.475783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:23.475783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:23.475783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:23.475783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:23.475783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:23.475783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:23.475783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:23.475783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:23.475783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:23.475783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:23.475783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:23.475783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:23.475783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:23.475783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:23.475783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:23.475783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:23.476660 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:23.476660 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:23.476660 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:23.476660 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:23.476660 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:23.476660 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:23.476660 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:23.476660 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:23.476660 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 371/742] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:23.477645 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458db7bd-70fb-44cb-8e99-46277c447041 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:23.477934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458db7bd-70fb-44cb-8e99-46277c447041 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:23.478206 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458db7bd-70fb-44cb-8e99-46277c447041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:23.492507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714decab-c681-4c8d-b201-605568aa69f2 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:23.495797 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-458db7bd-70fb-44cb-8e99-46277c447041 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:23.497008 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-458db7bd-70fb-44cb-8e99-46277c447041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:23.525947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d565660-6e5f-4f2a-922e-509cc9faf662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:23.527118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d565660-6e5f-4f2a-922e-509cc9faf662 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:23.527398 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d565660-6e5f-4f2a-922e-509cc9faf662 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:23.527671 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d565660-6e5f-4f2a-922e-509cc9faf662 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:23.534932 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d565660-6e5f-4f2a-922e-509cc9faf662 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:23.536388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d565660-6e5f-4f2a-922e-509cc9faf662 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:23.547475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-1d565660-6e5f-4f2a-922e-509cc9faf662 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 16:23:23.547475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:23.547475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:23.547475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:23.547475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:23.547475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:23.547475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:23.547475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:23.547475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:23.547475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:23.547475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:23.547475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:23.547475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:23.547475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:23.547475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:23.547475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:23.548532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:23.548532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:23.548532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:23.548532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:23.548532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:23.548532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:23.548532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:23.548532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:23.548532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:23.548532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:23.548532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:23.548532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:23.548532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:23.548532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:23.548532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:23.548532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:23.548532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:23.548532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:23.549997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:23.549997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:23.549997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:23.549997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:23.549997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:23.549997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:23.549997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:23.549997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:23.549997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 16:23:23.549997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:23.549997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 372/743] 10.222.0.74 () {56 vars in 1117 bytes} [Wed Jul 27 16:23:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:23.549997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d565660-6e5f-4f2a-922e-509cc9faf662 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:23.564756 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 372/744] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 16:23:23.565729 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458db7bd-70fb-44cb-8e99-46277c447041 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:23.621820 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:23.661018 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d565660-6e5f-4f2a-922e-509cc9faf662 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:23.664024 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458db7bd-70fb-44cb-8e99-46277c447041 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:23.665556 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:23.694610 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df098b-b62a-41c2-85bd-ada424b36a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:23.695452 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df098b-b62a-41c2-85bd-ada424b36a55 None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:23.695735 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df098b-b62a-41c2-85bd-ada424b36a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:23.696010 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df098b-b62a-41c2-85bd-ada424b36a55 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:23.696692 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df098b-b62a-41c2-85bd-ada424b36a55 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:23.698093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df098b-b62a-41c2-85bd-ada424b36a55 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:23.700046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead2dd0-0a76-4807-9641-5851bd8a8e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:23.700957 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead2dd0-0a76-4807-9641-5851bd8a8e46 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:23.701241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead2dd0-0a76-4807-9641-5851bd8a8e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:23.701518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead2dd0-0a76-4807-9641-5851bd8a8e46 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:23.702230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead2dd0-0a76-4807-9641-5851bd8a8e46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:23.702532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead2dd0-0a76-4807-9641-5851bd8a8e46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:23.704009 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead2dd0-0a76-4807-9641-5851bd8a8e46 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:23.712478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 373/745] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:23:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:23.713462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead2dd0-0a76-4807-9641-5851bd8a8e46 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:23.714375 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 373/746] 10.222.0.74 () {60 vars in 1141 bytes} [Wed Jul 27 16:23:23 2022] POST /identity/v3/services => generated 251 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:23:23.715047 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df098b-b62a-41c2-85bd-ada424b36a55 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:23.833460 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df098b-b62a-41c2-85bd-ada424b36a55 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:23.842880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead2dd0-0a76-4807-9641-5851bd8a8e46 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:23.854084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:23.876195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5a379-0b10-49c4-8e2c-5b27a90d0561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:23.877203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5a379-0b10-49c4-8e2c-5b27a90d0561 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:23.877523 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5a379-0b10-49c4-8e2c-5b27a90d0561 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:23.877742 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5a379-0b10-49c4-8e2c-5b27a90d0561 None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:23.878681 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5a379-0b10-49c4-8e2c-5b27a90d0561 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:23.879001 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5a379-0b10-49c4-8e2c-5b27a90d0561 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:23.880439 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5a379-0b10-49c4-8e2c-5b27a90d0561 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:23.889162 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838ae3f-6551-4c4f-9a58-3715c0514bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:23.889845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838ae3f-6551-4c4f-9a58-3715c0514bfb None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:23.890140 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838ae3f-6551-4c4f-9a58-3715c0514bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:23.890416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838ae3f-6551-4c4f-9a58-3715c0514bfb None admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:23.891429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838ae3f-6551-4c4f-9a58-3715c0514bfb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:23.891557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838ae3f-6551-4c4f-9a58-3715c0514bfb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:23.893411 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838ae3f-6551-4c4f-9a58-3715c0514bfb None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:23.901211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 374/747] 10.222.0.74 () {56 vars in 1149 bytes} [Wed Jul 27 16:23:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:23.902114 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5a379-0b10-49c4-8e2c-5b27a90d0561 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:23.905535 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 374/748] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:23.906869 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838ae3f-6551-4c4f-9a58-3715c0514bfb None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:23.937323 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:24.007433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5a379-0b10-49c4-8e2c-5b27a90d0561 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:24.022186 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838ae3f-6551-4c4f-9a58-3715c0514bfb None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:24.044656 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeaa8bd-4920-45cb-a053-b73b3783ba7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:24.045308 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeaa8bd-4920-45cb-a053-b73b3783ba7e None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:24.045588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeaa8bd-4920-45cb-a053-b73b3783ba7e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:24.045951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeaa8bd-4920-45cb-a053-b73b3783ba7e None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:24.046552 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeaa8bd-4920-45cb-a053-b73b3783ba7e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:24.047922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeaa8bd-4920-45cb-a053-b73b3783ba7e None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:24.061481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7afbfd-c699-4526-8e5e-74f8238c0cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:24.062126 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7afbfd-c699-4526-8e5e-74f8238c0cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:24.062407 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7afbfd-c699-4526-8e5e-74f8238c0cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:24.062704 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7afbfd-c699-4526-8e5e-74f8238c0cf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:24.065318 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 375/749] 10.222.0.74 () {60 vars in 1141 bytes} [Wed Jul 27 16:23:23 2022] POST /identity/v3/services => generated 240 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:23:24.066449 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeaa8bd-4920-45cb-a053-b73b3783ba7e None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:24.085654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7afbfd-c699-4526-8e5e-74f8238c0cf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:24.087825 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7afbfd-c699-4526-8e5e-74f8238c0cf0 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:24.111172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-0b7afbfd-c699-4526-8e5e-74f8238c0cf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:24.111172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:24.111172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:24.111172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:24.111172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:24.111172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:24.111172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:24.111172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:24.111172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:24.111172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:24.111172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:24.111172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:24.111172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:24.111172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:24.111172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:24.112138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:24.112138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:24.112138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:24.112138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:24.112138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:24.112138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:24.112138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:24.112138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:24.112138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:24.112138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:24.112138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:24.112138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:24.112138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:24.112138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:24.112138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:24.112138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:24.112138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:24.114440 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:24.114440 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:24.114440 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:24.114440 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:24.114440 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 375/750] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:24.115264 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7afbfd-c699-4526-8e5e-74f8238c0cf0 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:24.115509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:24.207493 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeaa8bd-4920-45cb-a053-b73b3783ba7e None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:24.236029 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:24.251971 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b047da-2cbf-4344-aed3-790c1f50ee35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:24.252613 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b047da-2cbf-4344-aed3-790c1f50ee35 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:24.252916 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b047da-2cbf-4344-aed3-790c1f50ee35 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:24.253216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b047da-2cbf-4344-aed3-790c1f50ee35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:24.273149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7afbfd-c699-4526-8e5e-74f8238c0cf0 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:24.328526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a1031-ff98-4f4f-af46-0a6a61b319ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:24.329233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a1031-ff98-4f4f-af46-0a6a61b319ce None admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:24.329519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a1031-ff98-4f4f-af46-0a6a61b319ce None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:24.329797 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a1031-ff98-4f4f-af46-0a6a61b319ce None admin] PATH_INFO: `/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/56dfafd15e284b7986ba07dd3819bf65/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:24.331792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a1031-ff98-4f4f-af46-0a6a61b319ce None admin] RBAC: Authorizing `identity:create_grant(project_id=8d6bcdd09cf24a5c8003951db37020cf, user_id=56dfafd15e284b7986ba07dd3819bf65, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:24.333273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a1031-ff98-4f4f-af46-0a6a61b319ce None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:24.345421 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b047da-2cbf-4344-aed3-790c1f50ee35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:24.347409 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b047da-2cbf-4344-aed3-790c1f50ee35 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:24.352573 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 376/751] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:23:24 2022] PUT /identity/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/56dfafd15e284b7986ba07dd3819bf65/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:23:24.353539 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a1031-ff98-4f4f-af46-0a6a61b319ce None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:24.402676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:24.544663 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 376/752] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:23:24 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 16:23:24.554539 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b047da-2cbf-4344-aed3-790c1f50ee35 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:24.572204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a1031-ff98-4f4f-af46-0a6a61b319ce None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:24.620914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374b603-4310-4620-a636-42a6cdb8abe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:24.620914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374b603-4310-4620-a636-42a6cdb8abe9 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:24.620914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374b603-4310-4620-a636-42a6cdb8abe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:24.620914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374b603-4310-4620-a636-42a6cdb8abe9 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:24.621380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374b603-4310-4620-a636-42a6cdb8abe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:24.621380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374b603-4310-4620-a636-42a6cdb8abe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:24.621838 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374b603-4310-4620-a636-42a6cdb8abe9 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:24.625336 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:24.640091 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 377/753] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:24.645765 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374b603-4310-4620-a636-42a6cdb8abe9 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:24.766970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b047da-2cbf-4344-aed3-790c1f50ee35 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:24.861827 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:24.871562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0401d14d-8c5f-43f3-972d-f1c8d1e745be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:24.872486 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401d14d-8c5f-43f3-972d-f1c8d1e745be None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:24.872767 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401d14d-8c5f-43f3-972d-f1c8d1e745be None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:24.873073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401d14d-8c5f-43f3-972d-f1c8d1e745be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:24.901698 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0401d14d-8c5f-43f3-972d-f1c8d1e745be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:24.904057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0401d14d-8c5f-43f3-972d-f1c8d1e745be None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:24.918122 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374b603-4310-4620-a636-42a6cdb8abe9 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:24.954370 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848db1f-dbfb-42c8-970b-d3a6f2974160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:24.955031 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848db1f-dbfb-42c8-970b-d3a6f2974160 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:24.955310 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848db1f-dbfb-42c8-970b-d3a6f2974160 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:24.955693 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848db1f-dbfb-42c8-970b-d3a6f2974160 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:24.965399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-0401d14d-8c5f-43f3-972d-f1c8d1e745be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:24.965399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:24.965399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:24.965399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:24.965399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:24.965399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:24.965399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:24.965399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:24.965399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:24.965399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:24.965399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:24.965399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:24.965399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:24.965399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:24.965399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:24.966350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:24.966350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:24.966350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:24.966350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:24.966350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:24.966350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:24.966350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:24.966350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:24.966350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:24.966350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:24.966350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:24.966350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:24.966350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:24.966350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:24.966350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:24.966350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:24.966350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:24.967824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:24.967824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:24.967824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:24.967824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:24.967824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 377/754] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:24.972518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1848db1f-dbfb-42c8-970b-d3a6f2974160 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:24.978065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0401d14d-8c5f-43f3-972d-f1c8d1e745be None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:24.981493 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1848db1f-dbfb-42c8-970b-d3a6f2974160 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:25.014115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-1848db1f-dbfb-42c8-970b-d3a6f2974160 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 16:23:25.014115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:25.014115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:25.014115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:25.014115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:25.014115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:25.014115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:25.014115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:25.014115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:25.014115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:25.014115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:25.014115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:25.014115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:25.014115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:25.014115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:25.014115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:25.015102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:25.015102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:25.015102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:25.015102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:25.015102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:25.015102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:25.015102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:25.015102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:25.015102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:25.015102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:25.015102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:25.015102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:25.015102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:25.015102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:25.015102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:25.015102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:25.015102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:25.015102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:25.017289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:25.017289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:25.017289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:25.017289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:25.017289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:25.017289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:25.017289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:25.017289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:25.017289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 16:23:25.017289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:25.017289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 378/755] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:23:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:25.026082 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848db1f-dbfb-42c8-970b-d3a6f2974160 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:25.143479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:25.202591 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848db1f-dbfb-42c8-970b-d3a6f2974160 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:25.249548 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13bbd7-c75e-41be-8743-6ca4575ff774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:25.251068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13bbd7-c75e-41be-8743-6ca4575ff774 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:25.251548 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13bbd7-c75e-41be-8743-6ca4575ff774 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:25.252017 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13bbd7-c75e-41be-8743-6ca4575ff774 None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:25.253040 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf13bbd7-c75e-41be-8743-6ca4575ff774 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:25.253573 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf13bbd7-c75e-41be-8743-6ca4575ff774 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:25.255315 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf13bbd7-c75e-41be-8743-6ca4575ff774 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:25.309375 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0401d14d-8c5f-43f3-972d-f1c8d1e745be None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:25.388944 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e66ac5-b29a-4cde-a869-7cf9618771c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:25.390006 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e66ac5-b29a-4cde-a869-7cf9618771c2 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:25.390514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e66ac5-b29a-4cde-a869-7cf9618771c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:25.390985 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e66ac5-b29a-4cde-a869-7cf9618771c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:25.391883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e66ac5-b29a-4cde-a869-7cf9618771c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:25.392393 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e66ac5-b29a-4cde-a869-7cf9618771c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:25.394089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e66ac5-b29a-4cde-a869-7cf9618771c2 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:25.424915 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 378/756] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:25.430689 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e66ac5-b29a-4cde-a869-7cf9618771c2 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:25.500219 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 379/757] 10.222.0.74 () {56 vars in 1135 bytes} [Wed Jul 27 16:23:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:25.503800 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13bbd7-c75e-41be-8743-6ca4575ff774 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:25.545245 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:25.658819 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13bbd7-c75e-41be-8743-6ca4575ff774 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:25.676227 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e66ac5-b29a-4cde-a869-7cf9618771c2 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:25.686709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:25.712613 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c056beb-0416-4a05-873c-4ab15ddbb4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:25.713246 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c056beb-0416-4a05-873c-4ab15ddbb4db None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:25.713516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c056beb-0416-4a05-873c-4ab15ddbb4db None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:25.714285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c056beb-0416-4a05-873c-4ab15ddbb4db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:25.736036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9c145-a61b-49fe-b1f8-1076b309041e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:25.736691 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9c145-a61b-49fe-b1f8-1076b309041e None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:25.736991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9c145-a61b-49fe-b1f8-1076b309041e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:25.737266 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9c145-a61b-49fe-b1f8-1076b309041e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:25.753320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9c145-a61b-49fe-b1f8-1076b309041e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:25.754842 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9c145-a61b-49fe-b1f8-1076b309041e None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:25.777341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-c2b9c145-a61b-49fe-b1f8-1076b309041e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:25.777341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:25.777341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:25.777341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:25.777341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:25.777341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:25.777341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:25.777341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:25.777341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:25.777341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:25.777341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:25.777341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:25.777341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:25.777341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:25.777341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:25.778279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:25.778279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:25.778279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:25.778279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:25.778279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:25.778279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:25.778279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:25.778279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:25.778279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:25.778279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:25.778279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:25.778279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:25.778279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:25.778279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:25.778279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:25.778279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:25.778279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:25.779142 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:25.779142 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:25.779142 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:25.779142 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:25.779142 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:25.779142 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:25.779142 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:25.779142 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:25.795817 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 379/758] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:23:25.797048 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9c145-a61b-49fe-b1f8-1076b309041e None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:25.867174 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c056beb-0416-4a05-873c-4ab15ddbb4db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:25.868617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c056beb-0416-4a05-873c-4ab15ddbb4db None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:25.953027 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9c145-a61b-49fe-b1f8-1076b309041e None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:25.955857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:25.991778 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2905b-7ac9-49e4-99fc-e0bb8be5d94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:25.992401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2905b-7ac9-49e4-99fc-e0bb8be5d94e None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:25.992676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2905b-7ac9-49e4-99fc-e0bb8be5d94e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:25.992970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2905b-7ac9-49e4-99fc-e0bb8be5d94e None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:25.994050 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2905b-7ac9-49e4-99fc-e0bb8be5d94e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:25.994361 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2905b-7ac9-49e4-99fc-e0bb8be5d94e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:25.996028 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2905b-7ac9-49e4-99fc-e0bb8be5d94e None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:26.015380 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 380/759] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:26.016640 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2905b-7ac9-49e4-99fc-e0bb8be5d94e None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:26.057705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 380/760] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:23:25 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 16:23:26.058997 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a590b-c04e-4327-9526-791185a2f194 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:26.059407 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a590b-c04e-4327-9526-791185a2f194 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:26.059689 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a590b-c04e-4327-9526-791185a2f194 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:26.061315 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 381/761] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:26.066076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a590b-c04e-4327-9526-791185a2f194 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:26.109242 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:26.132942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2905b-7ac9-49e4-99fc-e0bb8be5d94e None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:26.172183 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984550a8-6808-4018-96b6-6d98d2480d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:26.173146 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984550a8-6808-4018-96b6-6d98d2480d38 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:26.173574 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984550a8-6808-4018-96b6-6d98d2480d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:26.173991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984550a8-6808-4018-96b6-6d98d2480d38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:26.175042 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984550a8-6808-4018-96b6-6d98d2480d38 None admin] RBAC: Adding query filter params (role.id=2aa0cdfed849496db8f3692c9569d7ee, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=07d6bffc69674103b4c6540598a780b6) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:26.175517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984550a8-6808-4018-96b6-6d98d2480d38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:26.177234 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984550a8-6808-4018-96b6-6d98d2480d38 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:26.183881 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 381/762] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:23:26 2022] GET /identity/v3/role_assignments?user.id=07d6bffc69674103b4c6540598a780b6&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=2aa0cdfed849496db8f3692c9569d7ee => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:26.185310 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f67fee-f197-49b7-bb79-4ad673447f4a None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:26.185729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f67fee-f197-49b7-bb79-4ad673447f4a None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:26.186153 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f67fee-f197-49b7-bb79-4ad673447f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:26.219866 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-91f67fee-f197-49b7-bb79-4ad673447f4a None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:26.221280 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a590b-c04e-4327-9526-791185a2f194 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:26.224407 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-91f67fee-f197-49b7-bb79-4ad673447f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:26.268969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:26.301444 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a23042-08c1-4ddb-a540-04678367f7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:26.302707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a23042-08c1-4ddb-a540-04678367f7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:26.303203 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a23042-08c1-4ddb-a540-04678367f7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:26.303699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a23042-08c1-4ddb-a540-04678367f7d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:26.316051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a23042-08c1-4ddb-a540-04678367f7d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:26.318029 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a23042-08c1-4ddb-a540-04678367f7d3 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:26.334900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-90a23042-08c1-4ddb-a540-04678367f7d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:26.334900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:26.334900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:26.334900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:26.334900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:26.334900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:26.334900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:26.334900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:26.334900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:26.334900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:26.334900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:26.334900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:26.334900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:26.334900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:26.334900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:26.335867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:26.335867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:26.335867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:26.335867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:26.335867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:26.335867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:26.335867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:26.335867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:26.335867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:26.335867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:26.335867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:26.335867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:26.335867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:26.335867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:26.335867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:26.335867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:26.335867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:26.336753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:26.336753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:26.336753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:26.336753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:26.339342 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 382/763] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:26.346031 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a23042-08c1-4ddb-a540-04678367f7d3 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:26.363730 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 382/764] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:26.402155 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f67fee-f197-49b7-bb79-4ad673447f4a None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:26.423677 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:26.482545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a23042-08c1-4ddb-a540-04678367f7d3 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:26.554103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a2821-0ee6-4738-8e7f-d636d69b46b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:26.555111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a2821-0ee6-4738-8e7f-d636d69b46b5 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:26.555683 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a2821-0ee6-4738-8e7f-d636d69b46b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:26.556157 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a2821-0ee6-4738-8e7f-d636d69b46b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:26.557092 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a2821-0ee6-4738-8e7f-d636d69b46b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:26.558986 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a2821-0ee6-4738-8e7f-d636d69b46b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:26.560581 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a2821-0ee6-4738-8e7f-d636d69b46b5 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:26.572719 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f67fee-f197-49b7-bb79-4ad673447f4a None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:26.578438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 383/765] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:26.584174 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74738da2-682a-4f39-873f-5a33a5fb39e1 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:26.585267 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74738da2-682a-4f39-873f-5a33a5fb39e1 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:26.585777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74738da2-682a-4f39-873f-5a33a5fb39e1 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:26.587934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 384/766] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:26.589881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74738da2-682a-4f39-873f-5a33a5fb39e1 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:26.621105 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e828e8-67b6-4c9e-b26e-872069a1eb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:26.622086 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e828e8-67b6-4c9e-b26e-872069a1eb15 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:26.622409 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e828e8-67b6-4c9e-b26e-872069a1eb15 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:26.622884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e828e8-67b6-4c9e-b26e-872069a1eb15 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:26.623983 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e828e8-67b6-4c9e-b26e-872069a1eb15 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:26.625922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e828e8-67b6-4c9e-b26e-872069a1eb15 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:26.638503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-22e828e8-67b6-4c9e-b26e-872069a1eb15 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 16:23:26.638503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:26.638503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:23:26.638503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:23:26.638503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:23:26.638503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:23:26.638503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:23:26.638503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:26.638503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 16:23:26.638503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:26.638503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:26.638503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:26.638503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:26.638503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:26.638503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:26.638503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:26.639438 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:26.639438 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:26.639438 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:26.639438 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:26.639438 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:26.639438 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:26.639438 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:26.639438 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:23:26.639438 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:23:26.639438 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:23:26.639438 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:23:26.639438 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:26.639438 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:26.639438 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:26.639438 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:26.639438 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:26.640333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:26.640333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:26.640333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:26.640333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:26.640333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:26.640333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:26.640333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:26.640333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:26.640333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:26.640333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:26.640333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:26.640333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:23:26.640333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:26.640333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 16:23:26.640333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:26.640333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 383/767] 10.222.0.74 () {56 vars in 1129 bytes} [Wed Jul 27 16:23:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:26.641696 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d69292-6c68-48f8-b1e4-77760aec324c None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:26.641696 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d69292-6c68-48f8-b1e4-77760aec324c None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:26.641863 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d69292-6c68-48f8-b1e4-77760aec324c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:26.664830 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-32d69292-6c68-48f8-b1e4-77760aec324c None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:26.666695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-32d69292-6c68-48f8-b1e4-77760aec324c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:26.689751 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:26.721327 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74738da2-682a-4f39-873f-5a33a5fb39e1 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:26.749985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:26.754784 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 384/768] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:26.755728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d69292-6c68-48f8-b1e4-77760aec324c None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:26.768945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a514c34f-cec5-4a1c-9261-2484ad9dfbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:26.770016 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514c34f-cec5-4a1c-9261-2484ad9dfbc1 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:26.770588 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514c34f-cec5-4a1c-9261-2484ad9dfbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:26.770924 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514c34f-cec5-4a1c-9261-2484ad9dfbc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:26.779351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a514c34f-cec5-4a1c-9261-2484ad9dfbc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:26.780799 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a514c34f-cec5-4a1c-9261-2484ad9dfbc1 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:26.793188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-a514c34f-cec5-4a1c-9261-2484ad9dfbc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:26.793188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:26.793188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:26.793188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:26.793188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:26.793188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:26.793188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:26.793188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:26.793188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:26.793188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:26.793188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:26.793188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:26.793188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:26.793188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:26.793188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:26.794087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:26.794087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:26.794087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:26.794087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:26.794087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:26.794087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:26.794087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:26.794087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:26.794087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:26.794087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:26.794087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:26.794087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:26.794087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:26.794087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:26.794087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:26.794087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:26.794087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:26.794988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:26.794988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:26.794988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:26.794988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:26.794988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:26.794988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:26.794988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:26.794988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:26.794988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 385/769] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:26.795468 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27262024-e5bb-405d-8160-e94bd0879a38 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:26.796033 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27262024-e5bb-405d-8160-e94bd0879a38 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:26.796033 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27262024-e5bb-405d-8160-e94bd0879a38 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:26.797401 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 386/770] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:26.797936 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27262024-e5bb-405d-8160-e94bd0879a38 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:26.882549 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d69292-6c68-48f8-b1e4-77760aec324c None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:26.911380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27262024-e5bb-405d-8160-e94bd0879a38 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:26.918435 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5c1e0-4450-4417-b3b7-361b06c74203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:26.919060 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5c1e0-4450-4417-b3b7-361b06c74203 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:26.919341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5c1e0-4450-4417-b3b7-361b06c74203 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:26.919614 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5c1e0-4450-4417-b3b7-361b06c74203 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:26.920714 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5c1e0-4450-4417-b3b7-361b06c74203 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:26.921066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5c1e0-4450-4417-b3b7-361b06c74203 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:26.922800 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5c1e0-4450-4417-b3b7-361b06c74203 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:26.932244 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 385/771] 10.222.0.74 () {56 vars in 1124 bytes} [Wed Jul 27 16:23:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:26.932244 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5c1e0-4450-4417-b3b7-361b06c74203 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:26.950628 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:26.953974 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb3e8e-90e6-4ae3-a50c-c974b1420289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:26.954693 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb3e8e-90e6-4ae3-a50c-c974b1420289 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:26.954975 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb3e8e-90e6-4ae3-a50c-c974b1420289 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:26.955561 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb3e8e-90e6-4ae3-a50c-c974b1420289 None admin] PATH_INFO: `/v3/services/network` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:26.955854 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb3e8e-90e6-4ae3-a50c-c974b1420289 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:26.957264 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb3e8e-90e6-4ae3-a50c-c974b1420289 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:26.967270 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-8bfb3e8e-90e6-4ae3-a50c-c974b1420289 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 16:23:26.967270 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:26.967270 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:23:26.967270 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:23:26.967270 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:23:26.967270 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:23:26.967270 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:23:26.967270 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:26.967270 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 16:23:26.967270 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:26.967270 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:26.967270 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:26.967270 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:26.967270 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:26.967270 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:26.967270 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:26.967270 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:26.968276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:26.968276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:26.968276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:26.968276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:26.968276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:26.968276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:26.968276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:23:26.968276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:23:26.968276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:23:26.968276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:23:26.968276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:26.968276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:26.968276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:26.968276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:26.968276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:26.968276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:26.969137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:26.969137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:26.969137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:26.969137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:26.969137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:26.969137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:26.969137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:26.969137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:26.969137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:26.969137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:26.969137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:23:26.969137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:26.969137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 16:23:26.969137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:26.969137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 387/772] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:26 2022] GET /identity/v3/services/network => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:26.969932 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5d7b6-d878-40bd-87aa-1a6fefcbe5aa None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:26.969932 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5d7b6-d878-40bd-87aa-1a6fefcbe5aa None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:26.970052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5d7b6-d878-40bd-87aa-1a6fefcbe5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:26.989177 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-cef5d7b6-d878-40bd-87aa-1a6fefcbe5aa None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:26.990395 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-cef5d7b6-d878-40bd-87aa-1a6fefcbe5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:27.044561 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:27.076928 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 388/773] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:27.076928 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5d7b6-d878-40bd-87aa-1a6fefcbe5aa None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:27.167135 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5c1e0-4450-4417-b3b7-361b06c74203 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:27.192241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5d7b6-d878-40bd-87aa-1a6fefcbe5aa None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:27.202108 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a456b3-59e6-4c83-9153-3e9ff19fa33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:27.202626 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a456b3-59e6-4c83-9153-3e9ff19fa33f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:27.202906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a456b3-59e6-4c83-9153-3e9ff19fa33f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:27.203178 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a456b3-59e6-4c83-9153-3e9ff19fa33f None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:27.203916 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a456b3-59e6-4c83-9153-3e9ff19fa33f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:27.204218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a456b3-59e6-4c83-9153-3e9ff19fa33f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:27.205672 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a456b3-59e6-4c83-9153-3e9ff19fa33f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:27.222510 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 386/774] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:27.222510 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a456b3-59e6-4c83-9153-3e9ff19fa33f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:27.234609 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d40c4-095e-4e10-8ce2-6a10b60c748b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:27.235229 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d40c4-095e-4e10-8ce2-6a10b60c748b None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:27.235511 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d40c4-095e-4e10-8ce2-6a10b60c748b None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:27.235789 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d40c4-095e-4e10-8ce2-6a10b60c748b None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:27.236464 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d40c4-095e-4e10-8ce2-6a10b60c748b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:27.236767 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d40c4-095e-4e10-8ce2-6a10b60c748b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:27.238213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d40c4-095e-4e10-8ce2-6a10b60c748b None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:27.247599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 389/775] 10.222.0.74 () {56 vars in 1124 bytes} [Wed Jul 27 16:23:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:27.247599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d40c4-095e-4e10-8ce2-6a10b60c748b None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:27.295663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:27.302638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:27.327507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a456b3-59e6-4c83-9153-3e9ff19fa33f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:27.354071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d40c4-095e-4e10-8ce2-6a10b60c748b None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:27.365143 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4db1a-497d-416b-84a2-fe5a476c2365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:27.366169 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4db1a-497d-416b-84a2-fe5a476c2365 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:27.366457 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4db1a-497d-416b-84a2-fe5a476c2365 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:27.367016 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4db1a-497d-416b-84a2-fe5a476c2365 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:27.367698 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4db1a-497d-416b-84a2-fe5a476c2365 None admin] RBAC: Adding query filter params (name=network) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:27.368001 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4db1a-497d-416b-84a2-fe5a476c2365 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:27.370157 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4db1a-497d-416b-84a2-fe5a476c2365 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:27.380721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 387/776] 10.222.0.74 () {56 vars in 1112 bytes} [Wed Jul 27 16:23:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:27.380721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4db1a-497d-416b-84a2-fe5a476c2365 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:27.390648 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a97847-512f-4c32-ac93-e71c1d1cbfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:27.391307 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a97847-512f-4c32-ac93-e71c1d1cbfb5 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:27.391595 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a97847-512f-4c32-ac93-e71c1d1cbfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:27.391874 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a97847-512f-4c32-ac93-e71c1d1cbfb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:27.402883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a97847-512f-4c32-ac93-e71c1d1cbfb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:27.408842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-80a97847-512f-4c32-ac93-e71c1d1cbfb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:23:27.408842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:27.408842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:27.408842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:27.408842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:27.408842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:27.408842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:27.408842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:27.408842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:27.408842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:27.408842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:27.408842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:27.408842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:27.408842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:27.408842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:27.408842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:27.409967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:27.409967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:27.409967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:27.409967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:27.409967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:27.409967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:27.409967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:27.409967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:27.409967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:27.409967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:27.409967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:27.409967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:27.409967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:27.409967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:27.409967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:27.409967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:27.410784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:27.410784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:27.410784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:27.410784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:27.410784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:27.410784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:27.410784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:27.410784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:27.410784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:27.410784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:27.410784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:23:27.410784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:27.410784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 390/777] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:23:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:27.411542 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a97847-512f-4c32-ac93-e71c1d1cbfb5 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:27.498779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4db1a-497d-416b-84a2-fe5a476c2365 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:27.520766 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a97847-512f-4c32-ac93-e71c1d1cbfb5 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:27.523650 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:27.535317 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b0793-9e41-433c-a38a-aca84a20ea4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:27.535937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b0793-9e41-433c-a38a-aca84a20ea4f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:27.536219 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b0793-9e41-433c-a38a-aca84a20ea4f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:27.536491 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b0793-9e41-433c-a38a-aca84a20ea4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:27.537299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b0793-9e41-433c-a38a-aca84a20ea4f None admin] RBAC: Adding query filter params (role.id=ca37e43082fa40c6a88b12680a0ca94b, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=bb45ded9f75e4fa2973b564f83bdfaf2) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:27.537611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b0793-9e41-433c-a38a-aca84a20ea4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:27.539048 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b0793-9e41-433c-a38a-aca84a20ea4f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:27.545111 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 388/778] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:23:27 2022] GET /identity/v3/role_assignments?user.id=bb45ded9f75e4fa2973b564f83bdfaf2&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=ca37e43082fa40c6a88b12680a0ca94b => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:27.556784 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b0793-9e41-433c-a38a-aca84a20ea4f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:27.557169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137234cd-c32a-49bb-a5c8-182084f30ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:27.558126 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137234cd-c32a-49bb-a5c8-182084f30ebd None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:27.558628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137234cd-c32a-49bb-a5c8-182084f30ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:27.559055 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137234cd-c32a-49bb-a5c8-182084f30ebd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:27.559893 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137234cd-c32a-49bb-a5c8-182084f30ebd None admin] RBAC: Adding query filter params (interface=public, service_id=25c2a7079bc944fab7f6f26ac97d6efa, region_id=RegionOne) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:27.560211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137234cd-c32a-49bb-a5c8-182084f30ebd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:27.561670 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137234cd-c32a-49bb-a5c8-182084f30ebd None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:27.566003 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:27.578457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 391/779] 10.222.0.74 () {56 vars in 1251 bytes} [Wed Jul 27 16:23:27 2022] GET /identity/v3/endpoints?service_id=25c2a7079bc944fab7f6f26ac97d6efa&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:27.580360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137234cd-c32a-49bb-a5c8-182084f30ebd None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:27.679132 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b0793-9e41-433c-a38a-aca84a20ea4f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:27.723390 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137234cd-c32a-49bb-a5c8-182084f30ebd None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:27.725657 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac8674-ba9c-4e83-a6cb-243776badfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:27.726281 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac8674-ba9c-4e83-a6cb-243776badfd5 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:27.726557 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac8674-ba9c-4e83-a6cb-243776badfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:27.726827 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac8674-ba9c-4e83-a6cb-243776badfd5 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:27.727510 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac8674-ba9c-4e83-a6cb-243776badfd5 None admin] RBAC: Adding query filter params (type=network) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:27.727812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac8674-ba9c-4e83-a6cb-243776badfd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:27.729219 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac8674-ba9c-4e83-a6cb-243776badfd5 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:27.739181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 389/780] 10.222.0.74 () {56 vars in 1112 bytes} [Wed Jul 27 16:23:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:27.740228 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac8674-ba9c-4e83-a6cb-243776badfd5 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:27.765866 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8647b44-f2cf-4e00-bc7b-13e763b9d8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:27.766872 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8647b44-f2cf-4e00-bc7b-13e763b9d8a1 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:27.767362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8647b44-f2cf-4e00-bc7b-13e763b9d8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:27.767834 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8647b44-f2cf-4e00-bc7b-13e763b9d8a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:27.768738 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8647b44-f2cf-4e00-bc7b-13e763b9d8a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:27.769250 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8647b44-f2cf-4e00-bc7b-13e763b9d8a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:27.770830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8647b44-f2cf-4e00-bc7b-13e763b9d8a1 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:27.784238 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 392/781] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:23:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:27.785429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8647b44-f2cf-4e00-bc7b-13e763b9d8a1 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:27.817187 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:27.818416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:27.867853 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac8674-ba9c-4e83-a6cb-243776badfd5 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:27.900947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8647b44-f2cf-4e00-bc7b-13e763b9d8a1 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:27.911373 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd6045-9378-4745-a9db-c39e80248119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:27.912297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd6045-9378-4745-a9db-c39e80248119 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:27.912764 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd6045-9378-4745-a9db-c39e80248119 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:27.913242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd6045-9378-4745-a9db-c39e80248119 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:27.914102 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd6045-9378-4745-a9db-c39e80248119 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:27.915662 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd6045-9378-4745-a9db-c39e80248119 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:27.921805 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 390/782] 10.222.0.74 () {56 vars in 1087 bytes} [Wed Jul 27 16:23:27 2022] GET /identity/v3/services => generated 772 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:27.922934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd6045-9378-4745-a9db-c39e80248119 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:27.957884 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d984045-e945-45c9-ab5a-bc204d1e67f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:27.958889 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d984045-e945-45c9-ab5a-bc204d1e67f3 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:27.959359 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d984045-e945-45c9-ab5a-bc204d1e67f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:27.959886 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d984045-e945-45c9-ab5a-bc204d1e67f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:27.961965 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d984045-e945-45c9-ab5a-bc204d1e67f3 None admin] RBAC: Adding query filter params (interface=public, service_id=dd1480af982c44c7850911a1b4c304d5, region_id=RegionOne) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:27.962499 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d984045-e945-45c9-ab5a-bc204d1e67f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:27.965458 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d984045-e945-45c9-ab5a-bc204d1e67f3 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:27.980280 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 393/783] 10.222.0.74 () {56 vars in 1251 bytes} [Wed Jul 27 16:23:27 2022] GET /identity/v3/endpoints?service_id=dd1480af982c44c7850911a1b4c304d5&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:27.985666 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d984045-e945-45c9-ab5a-bc204d1e67f3 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:28.126928 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd6045-9378-4745-a9db-c39e80248119 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:28.157407 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:28.166724 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:28.167758 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d984045-e945-45c9-ab5a-bc204d1e67f3 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:28.182293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4696a-0507-4ca8-a022-1a7225ca477a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:28.182996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4696a-0507-4ca8-a022-1a7225ca477a None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:28.183579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4696a-0507-4ca8-a022-1a7225ca477a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:28.183861 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4696a-0507-4ca8-a022-1a7225ca477a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:28.204027 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4696a-0507-4ca8-a022-1a7225ca477a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:28.205666 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4696a-0507-4ca8-a022-1a7225ca477a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:28.212061 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a38cb-ba19-4b56-b118-2c1f28bfafc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:28.213132 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a38cb-ba19-4b56-b118-2c1f28bfafc6 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:28.213585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a38cb-ba19-4b56-b118-2c1f28bfafc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:28.214089 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a38cb-ba19-4b56-b118-2c1f28bfafc6 None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:28.215010 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a38cb-ba19-4b56-b118-2c1f28bfafc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:28.216524 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a38cb-ba19-4b56-b118-2c1f28bfafc6 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:28.221821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-58c4696a-0507-4ca8-a022-1a7225ca477a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:28.221821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:28.221821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:28.221821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:28.221821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:28.221821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:28.221821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:28.221821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:28.221821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:28.221821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:28.221821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:28.221821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:28.221821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:28.221821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:28.221821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:28.222874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:28.222874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:28.222874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:28.222874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:28.222874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:28.222874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:28.222874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:28.222874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:28.222874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:28.222874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:28.222874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:28.222874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:28.222874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:28.222874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:28.222874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:28.222874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:28.222874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:28.223951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:28.223951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:28.223951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:28.223951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:28.224997 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 394/784] 10.222.0.74 () {56 vars in 1087 bytes} [Wed Jul 27 16:23:27 2022] GET /identity/v3/services => generated 772 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:28.225323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 391/785] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:28.231818 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a38cb-ba19-4b56-b118-2c1f28bfafc6 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:28.367525 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbba6b-7273-4d3e-89bd-d4be1ce62f45 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:28.368031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbba6b-7273-4d3e-89bd-d4be1ce62f45 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:28.368108 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbba6b-7273-4d3e-89bd-d4be1ce62f45 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:28.369695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 392/786] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:28.374880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a5345-f945-4a81-9c8b-a32a5be47822 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:28.375167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a5345-f945-4a81-9c8b-a32a5be47822 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:28.375448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a5345-f945-4a81-9c8b-a32a5be47822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:28.383195 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a38cb-ba19-4b56-b118-2c1f28bfafc6 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:28.394934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-092a5345-f945-4a81-9c8b-a32a5be47822 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:28.396303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-092a5345-f945-4a81-9c8b-a32a5be47822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:28.452610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe3bf6-a96e-4ac7-95fa-54eaa0cafd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:28.453280 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe3bf6-a96e-4ac7-95fa-54eaa0cafd56 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:28.453563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe3bf6-a96e-4ac7-95fa-54eaa0cafd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:28.453838 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe3bf6-a96e-4ac7-95fa-54eaa0cafd56 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:28.454551 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe3bf6-a96e-4ac7-95fa-54eaa0cafd56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:28.454854 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe3bf6-a96e-4ac7-95fa-54eaa0cafd56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:28.456256 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe3bf6-a96e-4ac7-95fa-54eaa0cafd56 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:28.485516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 395/787] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:28.489511 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe3bf6-a96e-4ac7-95fa-54eaa0cafd56 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:28.509852 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 393/788] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 16:23:28.522220 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:28.537948 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a5345-f945-4a81-9c8b-a32a5be47822 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:28.627035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe3bf6-a96e-4ac7-95fa-54eaa0cafd56 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:28.666370 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55b4c2-c2e5-467e-9284-89a51fd7d590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:28.667379 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55b4c2-c2e5-467e-9284-89a51fd7d590 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:28.668033 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55b4c2-c2e5-467e-9284-89a51fd7d590 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:28.669956 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55b4c2-c2e5-467e-9284-89a51fd7d590 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:28.684687 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55b4c2-c2e5-467e-9284-89a51fd7d590 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:28.686480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55b4c2-c2e5-467e-9284-89a51fd7d590 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:28.706513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-7c55b4c2-c2e5-467e-9284-89a51fd7d590 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 16:23:28.706513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:28.706513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:28.706513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:28.706513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:28.706513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:28.706513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:28.706513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:28.706513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:28.706513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:28.706513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:28.706513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:28.706513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:28.706513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:28.706513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:28.706513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:28.707504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:28.707504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:28.707504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:28.707504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:28.707504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:28.707504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:28.707504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:28.707504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:28.707504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:28.707504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:28.707504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:28.707504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:28.707504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:28.707504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:28.707504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:28.707504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:28.707504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:28.707504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:28.708437 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:28.708437 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:28.708437 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:28.708437 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:28.708437 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:28.708437 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:28.708437 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:28.708437 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:28.708437 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 16:23:28.708437 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:28.714709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a5345-f945-4a81-9c8b-a32a5be47822 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:28.719096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 396/789] 10.222.0.74 () {56 vars in 1093 bytes} [Wed Jul 27 16:23:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:28.723741 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55b4c2-c2e5-467e-9284-89a51fd7d590 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:28.742768 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:28.784635 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f90db-15ec-4c1b-91b1-97248a5dd316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:28.787478 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f90db-15ec-4c1b-91b1-97248a5dd316 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:28.787762 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f90db-15ec-4c1b-91b1-97248a5dd316 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:28.788035 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f90db-15ec-4c1b-91b1-97248a5dd316 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:28.813028 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f90db-15ec-4c1b-91b1-97248a5dd316 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:28.817963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-570f90db-15ec-4c1b-91b1-97248a5dd316 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:28.817963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:28.817963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:28.817963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:28.817963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:28.817963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:28.817963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:28.817963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:28.817963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:28.817963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:28.817963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:28.817963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:28.817963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:28.817963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:28.817963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:28.817963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:28.818883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:28.818883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:28.818883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:28.818883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:28.818883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:28.818883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:28.818883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:28.818883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:28.818883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:28.818883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:28.818883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:28.818883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:28.818883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:28.818883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:28.818883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:28.818883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:28.819722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:28.819722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:28.819722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:28.819722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:28.819722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:28.819722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:28.819722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:28.819722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:28.819722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:28.819722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:28.819722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:28.819722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:28.827384 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 394/790] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:23:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:28.828251 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f90db-15ec-4c1b-91b1-97248a5dd316 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:28.879685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:28.925837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55b4c2-c2e5-467e-9284-89a51fd7d590 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:28.961890 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc04d1-ed6e-4c6a-9a84-dbe8184bd677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:28.962745 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc04d1-ed6e-4c6a-9a84-dbe8184bd677 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:28.963176 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc04d1-ed6e-4c6a-9a84-dbe8184bd677 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:28.963605 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc04d1-ed6e-4c6a-9a84-dbe8184bd677 None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:28.964595 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc04d1-ed6e-4c6a-9a84-dbe8184bd677 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:28.965102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc04d1-ed6e-4c6a-9a84-dbe8184bd677 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:28.966665 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc04d1-ed6e-4c6a-9a84-dbe8184bd677 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:28.985431 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 397/791] 10.222.0.74 () {56 vars in 1133 bytes} [Wed Jul 27 16:23:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:28.990932 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc04d1-ed6e-4c6a-9a84-dbe8184bd677 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:29.086825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f90db-15ec-4c1b-91b1-97248a5dd316 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:29.102455 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc04d1-ed6e-4c6a-9a84-dbe8184bd677 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:29.141502 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb5330-3310-492b-835c-ab2819c63dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:29.142522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb5330-3310-492b-835c-ab2819c63dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:29.143334 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb5330-3310-492b-835c-ab2819c63dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:29.143765 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb5330-3310-492b-835c-ab2819c63dd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:29.147746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:29.215356 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a937c5-ae6e-4788-9b27-426776886e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:29.215997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a937c5-ae6e-4788-9b27-426776886e4a None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:29.216283 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a937c5-ae6e-4788-9b27-426776886e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:29.216561 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a937c5-ae6e-4788-9b27-426776886e4a None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:29.217334 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a937c5-ae6e-4788-9b27-426776886e4a None admin] RBAC: Adding query filter params (name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:29.217644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a937c5-ae6e-4788-9b27-426776886e4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:29.219040 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a937c5-ae6e-4788-9b27-426776886e4a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:29.233584 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 395/792] 10.222.0.74 () {56 vars in 1103 bytes} [Wed Jul 27 16:23:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:29.234321 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a937c5-ae6e-4788-9b27-426776886e4a None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:29.236100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb5330-3310-492b-835c-ab2819c63dd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:29.237740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb5330-3310-492b-835c-ab2819c63dd3 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:29.351644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 398/793] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:23:28 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 16:23:29.357497 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb5330-3310-492b-835c-ab2819c63dd3 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:29.369606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:29.446532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a937c5-ae6e-4788-9b27-426776886e4a None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:29.474729 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:29.494868 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130bc72-8c43-49dc-a50b-7891fbbb0318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:29.496107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130bc72-8c43-49dc-a50b-7891fbbb0318 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:29.496329 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130bc72-8c43-49dc-a50b-7891fbbb0318 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:29.496605 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130bc72-8c43-49dc-a50b-7891fbbb0318 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:29.517218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130bc72-8c43-49dc-a50b-7891fbbb0318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:29.518668 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130bc72-8c43-49dc-a50b-7891fbbb0318 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:29.543229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-e130bc72-8c43-49dc-a50b-7891fbbb0318 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:29.543229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:29.543229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:29.543229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:29.543229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:29.543229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:29.543229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:29.543229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:29.543229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:29.543229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:29.543229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:29.543229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:29.543229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:29.543229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:29.543229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:29.544148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:29.544148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:29.544148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:29.544148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:29.544148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:29.544148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:29.544148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:29.544148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:29.544148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:29.544148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:29.544148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:29.544148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:29.544148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:29.544148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:29.544148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:29.544148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:29.544148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:29.545031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:29.545031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:29.545031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:29.545031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:29.548728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 396/794] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:29.553825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130bc72-8c43-49dc-a50b-7891fbbb0318 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:29.565361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb5330-3310-492b-835c-ab2819c63dd3 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:29.606672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4de9d-fb10-48d9-81e0-9da8cee13553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:29.607328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4de9d-fb10-48d9-81e0-9da8cee13553 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:29.607612 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4de9d-fb10-48d9-81e0-9da8cee13553 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:29.607889 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4de9d-fb10-48d9-81e0-9da8cee13553 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:29.624630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4de9d-fb10-48d9-81e0-9da8cee13553 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:29.628367 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4de9d-fb10-48d9-81e0-9da8cee13553 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:29.654045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-4fc4de9d-fb10-48d9-81e0-9da8cee13553 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:29.654045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:29.654045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:29.654045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:29.654045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:29.654045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:29.654045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:29.654045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:29.654045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:29.654045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:29.654045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:29.654045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:29.654045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:29.654045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:29.654045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:29.654995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:29.654995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:29.654995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:29.654995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:29.654995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:29.654995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:29.654995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:29.654995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:29.654995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:29.654995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:29.654995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:29.654995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:29.654995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:29.654995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:29.654995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:29.654995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:29.654995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:29.655880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:29.655880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:29.655880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:29.655880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:29.660929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 399/795] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:29.664713 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4de9d-fb10-48d9-81e0-9da8cee13553 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:29.764419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:29.818986 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130bc72-8c43-49dc-a50b-7891fbbb0318 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:29.823858 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:29.843203 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4de9d-fb10-48d9-81e0-9da8cee13553 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:29.862255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa1e62-bd99-42b9-8ced-a18a422cda98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:29.863605 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa1e62-bd99-42b9-8ced-a18a422cda98 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:29.864344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa1e62-bd99-42b9-8ced-a18a422cda98 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:29.864772 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa1e62-bd99-42b9-8ced-a18a422cda98 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:29.865901 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa1e62-bd99-42b9-8ced-a18a422cda98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:29.866377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa1e62-bd99-42b9-8ced-a18a422cda98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:29.868276 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa1e62-bd99-42b9-8ced-a18a422cda98 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:29.886048 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 397/796] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:29.887564 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc24753-ef78-41c2-b8a6-faa1ba46582d None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:29.888015 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc24753-ef78-41c2-b8a6-faa1ba46582d None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:29.888274 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff66d96-1254-4781-a250-97de2505df64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:29.888606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc24753-ef78-41c2-b8a6-faa1ba46582d None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:29.889108 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff66d96-1254-4781-a250-97de2505df64 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:29.889489 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff66d96-1254-4781-a250-97de2505df64 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:29.889883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff66d96-1254-4781-a250-97de2505df64 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:29.894520 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff66d96-1254-4781-a250-97de2505df64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:29.894520 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff66d96-1254-4781-a250-97de2505df64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:29.894520 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff66d96-1254-4781-a250-97de2505df64 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:29.895107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 398/797] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:29.896590 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc24753-ef78-41c2-b8a6-faa1ba46582d None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:29.912975 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 400/798] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:29.914452 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afa781-9ed6-4738-88bb-b72a71583211 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:29.914870 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afa781-9ed6-4738-88bb-b72a71583211 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:29.915303 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afa781-9ed6-4738-88bb-b72a71583211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:29.940262 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-c5afa781-9ed6-4738-88bb-b72a71583211 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:29.941879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-c5afa781-9ed6-4738-88bb-b72a71583211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:30.024921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc24753-ef78-41c2-b8a6-faa1ba46582d None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:30.036910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 401/799] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:30.036910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afa781-9ed6-4738-88bb-b72a71583211 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:30.098184 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67188b6-30bc-4f19-8d62-427ae5f319be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:30.099771 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67188b6-30bc-4f19-8d62-427ae5f319be None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:30.100336 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67188b6-30bc-4f19-8d62-427ae5f319be None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:30.100824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67188b6-30bc-4f19-8d62-427ae5f319be None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:30.111602 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67188b6-30bc-4f19-8d62-427ae5f319be None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:30.113353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67188b6-30bc-4f19-8d62-427ae5f319be None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:30.130821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-f67188b6-30bc-4f19-8d62-427ae5f319be None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 16:23:30.130821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:30.130821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:30.130821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:30.130821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:30.130821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:30.130821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:30.130821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:30.130821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:30.130821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:30.130821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:30.130821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:30.130821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:30.130821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:30.130821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:30.130821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:30.131855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:30.131855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:30.131855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:30.131855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:30.131855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:30.131855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:30.131855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:30.131855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:30.131855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:30.131855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:30.131855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:30.131855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:30.131855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:30.131855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:30.131855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:30.131855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:30.131855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:30.131855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:30.132759 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:30.132759 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:30.132759 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:30.132759 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:30.132759 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:30.132759 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:30.132759 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:30.132759 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:30.132759 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 16:23:30.132759 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:30.139602 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 399/800] 10.222.0.74 () {56 vars in 1117 bytes} [Wed Jul 27 16:23:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:30.139602 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67188b6-30bc-4f19-8d62-427ae5f319be None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:30.151707 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:30.159701 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:30.193654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afa781-9ed6-4738-88bb-b72a71583211 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:30.231242 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b317fd3-5bf8-4f39-ab41-eb71cce3c0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:30.232302 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b317fd3-5bf8-4f39-ab41-eb71cce3c0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:30.232802 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b317fd3-5bf8-4f39-ab41-eb71cce3c0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:30.233307 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b317fd3-5bf8-4f39-ab41-eb71cce3c0a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:30.240854 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b317fd3-5bf8-4f39-ab41-eb71cce3c0a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:30.242625 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b317fd3-5bf8-4f39-ab41-eb71cce3c0a3 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:30.246033 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67188b6-30bc-4f19-8d62-427ae5f319be None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:30.255402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-4b317fd3-5bf8-4f39-ab41-eb71cce3c0a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:30.255402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:30.255402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:30.255402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:30.255402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:30.255402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:30.255402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:30.255402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:30.255402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:30.255402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:30.255402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:30.255402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:30.255402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:30.255402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:30.255402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:30.256447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:30.256447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:30.256447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:30.256447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:30.256447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:30.256447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:30.256447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:30.256447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:30.256447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:30.256447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:30.256447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:30.256447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:30.256447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:30.256447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:30.256447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:30.256447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:30.256447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:30.257482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:30.257482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:30.257482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:30.257482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:30.257482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:30.257482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:30.257482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:30.257482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:30.257482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 402/801] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:30.257482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b317fd3-5bf8-4f39-ab41-eb71cce3c0a3 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:30.289283 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14cf369-d1d2-4300-8c6e-7311177b5221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:30.289847 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cf369-d1d2-4300-8c6e-7311177b5221 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:30.290125 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cf369-d1d2-4300-8c6e-7311177b5221 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:30.290400 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cf369-d1d2-4300-8c6e-7311177b5221 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:30.302432 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14cf369-d1d2-4300-8c6e-7311177b5221 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:30.308469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-d14cf369-d1d2-4300-8c6e-7311177b5221 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:23:30.308469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:30.308469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:30.308469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:30.308469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:30.308469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:30.308469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:30.308469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:30.308469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:30.308469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:30.308469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:30.308469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:30.308469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:30.308469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:30.308469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:30.308469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:30.309433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:30.309433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:30.309433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:30.309433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:30.309433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:30.309433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:30.309433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:30.309433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:30.309433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:30.309433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:30.309433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:30.309433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:30.309433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:30.309433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:30.309433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:30.309433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:30.310259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:30.310259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:30.310259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:30.310259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:30.310259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:30.310259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:30.310259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:30.310259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:30.310259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:30.310259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:30.310259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:23:30.310259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:30.310259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 400/802] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:23:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:30.311527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f137984-0929-4f08-9392-99fc906a0371 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:30.311527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f137984-0929-4f08-9392-99fc906a0371 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:30.311767 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f137984-0929-4f08-9392-99fc906a0371 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:30.313131 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 401/803] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:30.313731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f137984-0929-4f08-9392-99fc906a0371 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:30.384870 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b317fd3-5bf8-4f39-ab41-eb71cce3c0a3 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:30.389334 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:30.429496 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f137984-0929-4f08-9392-99fc906a0371 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:30.429928 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a9b36-5fa1-46f4-8595-9e7e2964e934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:30.430542 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a9b36-5fa1-46f4-8595-9e7e2964e934 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:30.430816 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a9b36-5fa1-46f4-8595-9e7e2964e934 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:30.431085 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a9b36-5fa1-46f4-8595-9e7e2964e934 None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:30.431863 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a9b36-5fa1-46f4-8595-9e7e2964e934 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:30.432163 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a9b36-5fa1-46f4-8595-9e7e2964e934 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:30.434243 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a9b36-5fa1-46f4-8595-9e7e2964e934 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:30.442766 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:30.455474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 403/804] 10.222.0.74 () {56 vars in 1149 bytes} [Wed Jul 27 16:23:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:30.456852 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5873b0-f94c-4117-a272-f7d8766bbad5 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:30.457243 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5873b0-f94c-4117-a272-f7d8766bbad5 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:30.457588 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5873b0-f94c-4117-a272-f7d8766bbad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:30.471769 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089f495-7e21-43ac-b43f-0f50cc1eeb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:30.472382 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089f495-7e21-43ac-b43f-0f50cc1eeb6b None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:30.472664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089f495-7e21-43ac-b43f-0f50cc1eeb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:30.472965 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089f495-7e21-43ac-b43f-0f50cc1eeb6b None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:30.473721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089f495-7e21-43ac-b43f-0f50cc1eeb6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:30.474013 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089f495-7e21-43ac-b43f-0f50cc1eeb6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:30.475446 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089f495-7e21-43ac-b43f-0f50cc1eeb6b None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:30.475819 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-ab5873b0-f94c-4117-a272-f7d8766bbad5 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:30.477019 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-ab5873b0-f94c-4117-a272-f7d8766bbad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:30.493067 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 402/805] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:30.493067 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089f495-7e21-43ac-b43f-0f50cc1eeb6b None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:30.558573 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 404/806] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:30.559333 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5873b0-f94c-4117-a272-f7d8766bbad5 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:30.606217 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089f495-7e21-43ac-b43f-0f50cc1eeb6b None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:30.633788 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:30.643120 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d8283-2c77-4d59-920f-2daa2c909c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:30.643743 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d8283-2c77-4d59-920f-2daa2c909c69 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:30.644023 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d8283-2c77-4d59-920f-2daa2c909c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:30.644303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d8283-2c77-4d59-920f-2daa2c909c69 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:30.645049 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d8283-2c77-4d59-920f-2daa2c909c69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:30.645359 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d8283-2c77-4d59-920f-2daa2c909c69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:30.646724 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d8283-2c77-4d59-920f-2daa2c909c69 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:30.655396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 403/807] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:23:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:30.656184 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d8283-2c77-4d59-920f-2daa2c909c69 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:30.670743 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5873b0-f94c-4117-a272-f7d8766bbad5 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:30.695871 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:30.705795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717e630-b57a-4951-8c1f-6b6e0b62b34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:30.706438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717e630-b57a-4951-8c1f-6b6e0b62b34e None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:30.706723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717e630-b57a-4951-8c1f-6b6e0b62b34e None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:30.706991 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717e630-b57a-4951-8c1f-6b6e0b62b34e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:30.717790 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717e630-b57a-4951-8c1f-6b6e0b62b34e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:30.719247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717e630-b57a-4951-8c1f-6b6e0b62b34e None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:30.733628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-8717e630-b57a-4951-8c1f-6b6e0b62b34e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:30.733628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:30.733628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:30.733628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:30.733628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:30.733628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:30.733628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:30.733628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:30.733628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:30.733628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:30.733628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:30.733628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:30.733628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:30.733628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:30.733628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:30.734512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:30.734512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:30.734512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:30.734512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:30.734512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:30.734512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:30.734512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:30.734512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:30.734512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:30.734512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:30.734512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:30.734512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:30.734512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:30.734512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:30.734512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:30.734512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:30.734512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:30.735390 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:30.735390 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:30.735390 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:30.735390 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:30.735390 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 405/808] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:30.735667 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717e630-b57a-4951-8c1f-6b6e0b62b34e None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:30.761044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d8283-2c77-4d59-920f-2daa2c909c69 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:30.796912 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695784f-7302-4752-96e4-c996f30de34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:30.797517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695784f-7302-4752-96e4-c996f30de34b None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:30.797793 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695784f-7302-4752-96e4-c996f30de34b None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:30.798061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695784f-7302-4752-96e4-c996f30de34b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:30.798831 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3695784f-7302-4752-96e4-c996f30de34b None admin] RBAC: Adding query filter params (role.id=ca37e43082fa40c6a88b12680a0ca94b, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=56dfafd15e284b7986ba07dd3819bf65) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:30.799130 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3695784f-7302-4752-96e4-c996f30de34b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:30.800537 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3695784f-7302-4752-96e4-c996f30de34b None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:30.806204 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 404/809] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:23:30 2022] GET /identity/v3/role_assignments?user.id=56dfafd15e284b7986ba07dd3819bf65&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=ca37e43082fa40c6a88b12680a0ca94b => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:30.807357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d89625-0ee3-4699-bff3-5c3641733610 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:30.808130 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d89625-0ee3-4699-bff3-5c3641733610 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:30.808403 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d89625-0ee3-4699-bff3-5c3641733610 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:30.809713 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 405/810] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:30.810316 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d89625-0ee3-4699-bff3-5c3641733610 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:30.842754 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717e630-b57a-4951-8c1f-6b6e0b62b34e None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:30.877013 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:30.881215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ff32e-2e24-43a7-a8c9-879848fdab71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:30.881919 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ff32e-2e24-43a7-a8c9-879848fdab71 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:30.882204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ff32e-2e24-43a7-a8c9-879848fdab71 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:30.882474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ff32e-2e24-43a7-a8c9-879848fdab71 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:30.883104 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ff32e-2e24-43a7-a8c9-879848fdab71 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:30.884507 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ff32e-2e24-43a7-a8c9-879848fdab71 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:30.894173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-e07ff32e-2e24-43a7-a8c9-879848fdab71 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 16:23:30.894173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:30.894173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:23:30.894173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:23:30.894173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:23:30.894173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:23:30.894173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:23:30.894173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:30.894173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 16:23:30.894173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:30.894173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:30.894173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:30.894173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:30.894173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:30.894173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:30.894173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:30.895226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:30.895226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:30.895226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:30.895226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:30.895226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:30.895226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:30.895226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:30.895226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:23:30.895226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:23:30.895226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:23:30.895226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:23:30.895226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:30.895226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:30.895226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:30.895226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:30.895226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:30.896096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:30.896096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:30.896096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:30.896096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:30.896096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:30.896096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:30.896096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:30.896096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:30.896096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:30.896096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:30.896096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:30.896096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:23:30.896096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:30.896096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 16:23:30.896096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:30.896096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 406/811] 10.222.0.74 () {56 vars in 1129 bytes} [Wed Jul 27 16:23:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:30.897671 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ff32e-2e24-43a7-a8c9-879848fdab71 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:30.934349 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d89625-0ee3-4699-bff3-5c3641733610 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:30.943823 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:30.976031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bbe5f-7131-4ee9-a458-7feb0f59b28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:30.976031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bbe5f-7131-4ee9-a458-7feb0f59b28f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:30.976350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bbe5f-7131-4ee9-a458-7feb0f59b28f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:30.976585 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bbe5f-7131-4ee9-a458-7feb0f59b28f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:30.991858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3bbe5f-7131-4ee9-a458-7feb0f59b28f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:30.993338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3bbe5f-7131-4ee9-a458-7feb0f59b28f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:31.011498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-fc3bbe5f-7131-4ee9-a458-7feb0f59b28f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:31.011498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:31.011498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:31.011498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:31.011498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:31.011498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:31.011498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:31.011498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:31.011498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:31.011498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:31.011498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:31.011498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:31.011498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:31.011498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:31.011498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:31.012425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:31.012425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:31.012425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:31.012425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:31.012425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:31.012425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:31.012425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:31.012425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:31.012425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:31.012425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:31.012425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:31.012425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:31.012425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:31.012425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:31.012425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:31.012425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:31.012425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:31.015216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:31.015216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:31.015216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:31.015216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:31.015568 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ff32e-2e24-43a7-a8c9-879848fdab71 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:31.015658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 406/812] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:31.015658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabe714-1c4f-4437-9efb-dfe681bdae0c None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:31.015793 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabe714-1c4f-4437-9efb-dfe681bdae0c None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:31.015993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabe714-1c4f-4437-9efb-dfe681bdae0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:31.034510 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-9fabe714-1c4f-4437-9efb-dfe681bdae0c None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:31.035686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9fabe714-1c4f-4437-9efb-dfe681bdae0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:31.053604 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c9ee21-2e5b-47bf-a557-39e5b6b7e114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:31.054226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9ee21-2e5b-47bf-a557-39e5b6b7e114 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:31.054507 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9ee21-2e5b-47bf-a557-39e5b6b7e114 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:31.054779 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9ee21-2e5b-47bf-a557-39e5b6b7e114 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:31.055463 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c9ee21-2e5b-47bf-a557-39e5b6b7e114 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:31.055768 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c9ee21-2e5b-47bf-a557-39e5b6b7e114 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:31.057164 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c9ee21-2e5b-47bf-a557-39e5b6b7e114 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:31.069887 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 407/813] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:31.070636 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c9ee21-2e5b-47bf-a557-39e5b6b7e114 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:31.115203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 407/814] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:31.116374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fabe714-1c4f-4437-9efb-dfe681bdae0c None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:31.169774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:31.183157 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c9ee21-2e5b-47bf-a557-39e5b6b7e114 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:31.184841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:31.220192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fa6fd-3d5b-4ada-8c9c-23df6b819cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:31.220807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fa6fd-3d5b-4ada-8c9c-23df6b819cae None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:31.221113 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fa6fd-3d5b-4ada-8c9c-23df6b819cae None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:31.221390 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fa6fd-3d5b-4ada-8c9c-23df6b819cae None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:31.222066 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9fa6fd-3d5b-4ada-8c9c-23df6b819cae None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:31.222373 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9fa6fd-3d5b-4ada-8c9c-23df6b819cae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:31.223736 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9fa6fd-3d5b-4ada-8c9c-23df6b819cae None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:31.228785 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 408/815] 10.222.0.74 () {56 vars in 1124 bytes} [Wed Jul 27 16:23:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:31.230458 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fa6fd-3d5b-4ada-8c9c-23df6b819cae None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:31.238905 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fabe714-1c4f-4437-9efb-dfe681bdae0c None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:31.275349 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff9198-9031-4b36-a5bd-f0b6ad431772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:31.275965 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff9198-9031-4b36-a5bd-f0b6ad431772 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:31.276797 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff9198-9031-4b36-a5bd-f0b6ad431772 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:31.277107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff9198-9031-4b36-a5bd-f0b6ad431772 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:31.277795 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff9198-9031-4b36-a5bd-f0b6ad431772 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:31.278092 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff9198-9031-4b36-a5bd-f0b6ad431772 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:31.279463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff9198-9031-4b36-a5bd-f0b6ad431772 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:31.295004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 408/816] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:31.295004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff9198-9031-4b36-a5bd-f0b6ad431772 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:31.348485 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fa6fd-3d5b-4ada-8c9c-23df6b819cae None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:31.383812 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa9d40-5bd2-4e24-bc7a-5e26cea4d410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:31.384566 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa9d40-5bd2-4e24-bc7a-5e26cea4d410 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:31.384843 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa9d40-5bd2-4e24-bc7a-5e26cea4d410 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:31.385167 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa9d40-5bd2-4e24-bc7a-5e26cea4d410 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:31.388069 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:31.394107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff9198-9031-4b36-a5bd-f0b6ad431772 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:31.395878 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa9d40-5bd2-4e24-bc7a-5e26cea4d410 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:31.397499 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa9d40-5bd2-4e24-bc7a-5e26cea4d410 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:31.410006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-6eaa9d40-5bd2-4e24-bc7a-5e26cea4d410 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:31.410006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:31.410006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:31.410006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:31.410006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:31.410006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:31.410006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:31.410006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:31.410006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:31.410006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:31.410006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:31.410006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:31.410006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:31.410006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:31.410006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:31.410879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:31.410879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:31.410879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:31.410879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:31.410879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:31.410879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:31.410879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:31.410879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:31.410879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:31.410879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:31.410879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:31.410879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:31.410879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:31.410879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:31.410879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:31.410879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:31.410879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:31.411743 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:31.411743 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:31.411743 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:31.411743 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:31.411743 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:31.411743 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:31.411743 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:31.411743 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:31.411743 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 409/817] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:31.412217 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa9d40-5bd2-4e24-bc7a-5e26cea4d410 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:31.431170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13aaa5f-f36c-4e3c-ad6f-d9698bfc132a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:31.431807 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13aaa5f-f36c-4e3c-ad6f-d9698bfc132a None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:31.432085 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13aaa5f-f36c-4e3c-ad6f-d9698bfc132a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:31.432355 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13aaa5f-f36c-4e3c-ad6f-d9698bfc132a None admin] PATH_INFO: `/v3/services/network` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:31.433045 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13aaa5f-f36c-4e3c-ad6f-d9698bfc132a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:31.434396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13aaa5f-f36c-4e3c-ad6f-d9698bfc132a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:31.442215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:31.443440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-b13aaa5f-f36c-4e3c-ad6f-d9698bfc132a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 16:23:31.443440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:31.443440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:23:31.443440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:23:31.443440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:23:31.443440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:23:31.443440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:23:31.443440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:31.443440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 16:23:31.443440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:31.443440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:31.443440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:31.443440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:31.443440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:31.443440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:31.443440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:31.443440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:31.444408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:31.444408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:31.444408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:31.444408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:31.444408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:31.444408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:31.444408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:23:31.444408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:23:31.444408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:23:31.444408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:23:31.444408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:31.444408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:31.444408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:31.444408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:31.444408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:31.444408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:31.445337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:31.445337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:31.445337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:31.445337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:31.445337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:31.445337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:31.445337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:31.445337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:31.445337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:31.445337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:31.445337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:23:31.445337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:31.445337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 16:23:31.445337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:31.445337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 409/818] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:31 2022] GET /identity/v3/services/network => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:31.446201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13aaa5f-f36c-4e3c-ad6f-d9698bfc132a None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:31.521768 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa9d40-5bd2-4e24-bc7a-5e26cea4d410 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:31.550161 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13aaa5f-f36c-4e3c-ad6f-d9698bfc132a None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:31.556910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea1d69-91f9-4286-8037-0fe144f23ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:31.558056 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea1d69-91f9-4286-8037-0fe144f23ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:31.558335 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea1d69-91f9-4286-8037-0fe144f23ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:31.558605 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea1d69-91f9-4286-8037-0fe144f23ea1 None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:31.559263 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea1d69-91f9-4286-8037-0fe144f23ea1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:31.559565 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea1d69-91f9-4286-8037-0fe144f23ea1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:31.560942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea1d69-91f9-4286-8037-0fe144f23ea1 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:31.570194 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 410/819] 10.222.0.74 () {56 vars in 1124 bytes} [Wed Jul 27 16:23:31 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:31.570194 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea1d69-91f9-4286-8037-0fe144f23ea1 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:31.589379 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a9ba7-4e52-4274-b153-bd64c38363da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:31.590030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a9ba7-4e52-4274-b153-bd64c38363da None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:31.590306 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a9ba7-4e52-4274-b153-bd64c38363da None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:31.590576 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a9ba7-4e52-4274-b153-bd64c38363da None admin] PATH_INFO: `/v3/users/swift` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:31.598150 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a9ba7-4e52-4274-b153-bd64c38363da None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:31.599641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a9ba7-4e52-4274-b153-bd64c38363da None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:31.610750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-423a9ba7-4e52-4274-b153-bd64c38363da None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 16:23:31.610750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:31.610750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:31.610750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:31.610750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:31.610750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:31.610750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:31.610750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:31.610750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:31.610750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:31.610750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:31.610750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:31.610750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:31.610750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:31.610750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:31.610750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:31.611833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:31.611833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:31.611833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:31.611833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:31.611833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:31.611833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:31.611833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:31.611833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:31.611833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:31.611833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:31.611833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:31.611833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:31.611833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:31.611833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:31.611833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:31.611833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:31.611833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:31.611833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:31.612754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:31.612754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:31.612754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:31.612754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:31.612754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:31.612754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:31.612754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:31.612754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:31.612754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 16:23:31.612754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:31.612754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 410/820] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:23:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:31.612754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a9ba7-4e52-4274-b153-bd64c38363da None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:31.637193 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:31.669961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:31.681216 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea1d69-91f9-4286-8037-0fe144f23ea1 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:31.722225 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c09388-58ff-46d7-abd7-a8f15f6c73e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:31.722847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c09388-58ff-46d7-abd7-a8f15f6c73e9 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:31.723129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c09388-58ff-46d7-abd7-a8f15f6c73e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:31.723405 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c09388-58ff-46d7-abd7-a8f15f6c73e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:31.724151 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c09388-58ff-46d7-abd7-a8f15f6c73e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:31.724458 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c09388-58ff-46d7-abd7-a8f15f6c73e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:31.725911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c09388-58ff-46d7-abd7-a8f15f6c73e9 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:31.727402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a9ba7-4e52-4274-b153-bd64c38363da None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:31.738150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 411/821] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:31.742220 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c09388-58ff-46d7-abd7-a8f15f6c73e9 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:31.762788 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0b2b1-cf2f-4f03-b1da-5640750395c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:31.763453 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0b2b1-cf2f-4f03-b1da-5640750395c6 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:31.763686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0b2b1-cf2f-4f03-b1da-5640750395c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:31.763961 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0b2b1-cf2f-4f03-b1da-5640750395c6 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:31.764636 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0b2b1-cf2f-4f03-b1da-5640750395c6 None admin] RBAC: Adding query filter params (name=network) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:31.764967 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0b2b1-cf2f-4f03-b1da-5640750395c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:31.766343 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0b2b1-cf2f-4f03-b1da-5640750395c6 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:31.771436 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 411/822] 10.222.0.74 () {56 vars in 1112 bytes} [Wed Jul 27 16:23:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:31.772052 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0b2b1-cf2f-4f03-b1da-5640750395c6 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:31.853642 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c09388-58ff-46d7-abd7-a8f15f6c73e9 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:31.861056 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:31.880403 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0b2b1-cf2f-4f03-b1da-5640750395c6 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:31.891951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba71d75-4f4d-4894-a589-6723223e6475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:31.892734 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba71d75-4f4d-4894-a589-6723223e6475 None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:31.893050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba71d75-4f4d-4894-a589-6723223e6475 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:31.893327 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba71d75-4f4d-4894-a589-6723223e6475 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:31.894028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba71d75-4f4d-4894-a589-6723223e6475 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:31.895386 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba71d75-4f4d-4894-a589-6723223e6475 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:31.917993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887c490-ca80-4c87-862b-6ace31d42972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:31.918334 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 412/823] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:23:31 2022] POST /identity/v3/endpoints => generated 346 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:23:31.918662 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887c490-ca80-4c87-862b-6ace31d42972 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:31.918938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887c490-ca80-4c87-862b-6ace31d42972 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:31.919208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887c490-ca80-4c87-862b-6ace31d42972 None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:31.920006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba71d75-4f4d-4894-a589-6723223e6475 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:31.920127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887c490-ca80-4c87-862b-6ace31d42972 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:31.920395 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887c490-ca80-4c87-862b-6ace31d42972 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:31.921936 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887c490-ca80-4c87-862b-6ace31d42972 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:31.934900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:31.948798 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 412/824] 10.222.0.74 () {56 vars in 1135 bytes} [Wed Jul 27 16:23:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:31.949881 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887c490-ca80-4c87-862b-6ace31d42972 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:32.076296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba71d75-4f4d-4894-a589-6723223e6475 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:32.098351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887c490-ca80-4c87-862b-6ace31d42972 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:32.130180 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f5e55-2570-449d-9d69-45197ee9336d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:32.131516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f5e55-2570-449d-9d69-45197ee9336d None admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:32.132146 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f5e55-2570-449d-9d69-45197ee9336d None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:32.132778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f5e55-2570-449d-9d69-45197ee9336d None admin] PATH_INFO: `/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/07d6bffc69674103b4c6540598a780b6/roles/2aa0cdfed849496db8f3692c9569d7ee` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:32.142881 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29cd67-8724-46c2-9494-a084cb9cae40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:32.143503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29cd67-8724-46c2-9494-a084cb9cae40 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:32.143779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29cd67-8724-46c2-9494-a084cb9cae40 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:32.144049 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29cd67-8724-46c2-9494-a084cb9cae40 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:32.144722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29cd67-8724-46c2-9494-a084cb9cae40 None admin] RBAC: Adding query filter params (type=network) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:32.145056 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29cd67-8724-46c2-9494-a084cb9cae40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:32.146429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29cd67-8724-46c2-9494-a084cb9cae40 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:32.152811 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 413/825] 10.222.0.74 () {56 vars in 1112 bytes} [Wed Jul 27 16:23:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:32.152811 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29cd67-8724-46c2-9494-a084cb9cae40 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:32.161284 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f5e55-2570-449d-9d69-45197ee9336d None admin] RBAC: Authorizing `identity:create_grant(project_id=8d6bcdd09cf24a5c8003951db37020cf, user_id=07d6bffc69674103b4c6540598a780b6, role_id=2aa0cdfed849496db8f3692c9569d7ee)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:32.162738 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f5e55-2570-449d-9d69-45197ee9336d None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:32.170798 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:32.183616 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 413/826] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:23:31 2022] PUT /identity/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/07d6bffc69674103b4c6540598a780b6/roles/2aa0cdfed849496db8f3692c9569d7ee => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:23:32.189021 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f5e55-2570-449d-9d69-45197ee9336d None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:32.295858 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:32.369583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29cd67-8724-46c2-9494-a084cb9cae40 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:32.390131 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f5e55-2570-449d-9d69-45197ee9336d None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:32.429586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec244f2-9329-4c3d-ac3d-c075606b91b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:32.430209 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec244f2-9329-4c3d-ac3d-c075606b91b6 None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:32.430490 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec244f2-9329-4c3d-ac3d-c075606b91b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:32.430767 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec244f2-9329-4c3d-ac3d-c075606b91b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:32.431471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec244f2-9329-4c3d-ac3d-c075606b91b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:32.433405 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec244f2-9329-4c3d-ac3d-c075606b91b6 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:32.435288 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbafff8-12b5-48a7-8e20-dc7376cd5b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:32.436513 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbafff8-12b5-48a7-8e20-dc7376cd5b0a None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:32.437185 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbafff8-12b5-48a7-8e20-dc7376cd5b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:32.437643 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbafff8-12b5-48a7-8e20-dc7376cd5b0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:32.452978 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbafff8-12b5-48a7-8e20-dc7376cd5b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:32.454820 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbafff8-12b5-48a7-8e20-dc7376cd5b0a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:32.457925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 414/827] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:23:32 2022] POST /identity/v3/endpoints => generated 327 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:23:32.482777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-8bbafff8-12b5-48a7-8e20-dc7376cd5b0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:32.482777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:32.482777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:32.482777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:32.482777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:32.482777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:32.482777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:32.482777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:32.482777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:32.482777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:32.482777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:32.482777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:32.482777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:32.482777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:32.482777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:32.483857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:32.483857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:32.483857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:32.483857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:32.483857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:32.483857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:32.483857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:32.483857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:32.483857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:32.483857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:32.483857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:32.483857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:32.483857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:32.483857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:32.483857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:32.483857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:32.483857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:32.484948 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:32.484948 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:32.484948 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:32.484948 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:32.486895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 414/828] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:32.492454 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec244f2-9329-4c3d-ac3d-c075606b91b6 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:32.633879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec244f2-9329-4c3d-ac3d-c075606b91b6 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:32.637505 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:32.685178 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6788b-1bd8-49a2-b5b3-13815c1eaf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:32.685728 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6788b-1bd8-49a2-b5b3-13815c1eaf2e None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:32.686059 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6788b-1bd8-49a2-b5b3-13815c1eaf2e None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:32.686339 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6788b-1bd8-49a2-b5b3-13815c1eaf2e None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:32.687037 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6788b-1bd8-49a2-b5b3-13815c1eaf2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:32.687342 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6788b-1bd8-49a2-b5b3-13815c1eaf2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:32.688720 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6788b-1bd8-49a2-b5b3-13815c1eaf2e None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:32.701351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 415/829] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:32.706598 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbafff8-12b5-48a7-8e20-dc7376cd5b0a None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:32.809397 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:32.859099 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbafff8-12b5-48a7-8e20-dc7376cd5b0a None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:32.897899 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f66ca-67cc-4699-8715-87098b843cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:32.898556 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f66ca-67cc-4699-8715-87098b843cda None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:32.898839 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f66ca-67cc-4699-8715-87098b843cda None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:32.899117 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f66ca-67cc-4699-8715-87098b843cda None admin] PATH_INFO: `/v3/projects/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:32.908460 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f66ca-67cc-4699-8715-87098b843cda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:32.909970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f66ca-67cc-4699-8715-87098b843cda None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:32.924170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-573f66ca-67cc-4699-8715-87098b843cda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:32.924170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:32.924170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:32.924170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:32.924170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:32.924170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:32.924170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:32.924170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:32.924170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:32.924170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:32.924170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:32.924170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:32.924170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:32.924170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:32.924170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:32.925084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:32.925084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:32.925084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:32.925084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:32.925084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:32.925084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:32.925084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:32.925084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:32.925084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:32.925084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:32.925084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:32.925084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:32.925084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:32.925084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:32.925084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:32.925084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:32.925084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:32.925967 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:32.925967 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:32.925967 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:32.925967 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:32.925967 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:32.925967 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:32.925967 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:32.925967 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:32.925967 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 415/830] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:32.931136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6788b-1bd8-49a2-b5b3-13815c1eaf2e None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:32.944846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e6cd1-1844-4ad2-9d3d-a1aa434304a9 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:32.945133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e6cd1-1844-4ad2-9d3d-a1aa434304a9 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:32.945359 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e6cd1-1844-4ad2-9d3d-a1aa434304a9 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:32.946826 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 416/831] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:32.953102 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16abb63-f1d1-4a40-82cb-3b3def595434 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:32.953386 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16abb63-f1d1-4a40-82cb-3b3def595434 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:32.953664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16abb63-f1d1-4a40-82cb-3b3def595434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:32.975749 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-f16abb63-f1d1-4a40-82cb-3b3def595434 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:32.977030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-f16abb63-f1d1-4a40-82cb-3b3def595434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:33.056915 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 417/832] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:33.072485 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6788b-1bd8-49a2-b5b3-13815c1eaf2e None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:33.079886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16abb63-f1d1-4a40-82cb-3b3def595434 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:33.117143 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873387d-0a90-4cb1-b8cf-a20f7985c371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:33.118191 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873387d-0a90-4cb1-b8cf-a20f7985c371 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:33.118683 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873387d-0a90-4cb1-b8cf-a20f7985c371 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:33.119168 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873387d-0a90-4cb1-b8cf-a20f7985c371 None admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:33.120690 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6873387d-0a90-4cb1-b8cf-a20f7985c371 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:33.121239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6873387d-0a90-4cb1-b8cf-a20f7985c371 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:33.122998 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6873387d-0a90-4cb1-b8cf-a20f7985c371 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:33.139936 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:33.144311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 416/833] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:33.156679 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873387d-0a90-4cb1-b8cf-a20f7985c371 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:33.187378 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:33.208415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16abb63-f1d1-4a40-82cb-3b3def595434 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:33.252346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd593f7-9500-4be9-bc11-b63d67bdd3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:33.253018 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd593f7-9500-4be9-bc11-b63d67bdd3af None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:33.253302 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd593f7-9500-4be9-bc11-b63d67bdd3af None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:33.253577 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd593f7-9500-4be9-bc11-b63d67bdd3af None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:33.254228 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd593f7-9500-4be9-bc11-b63d67bdd3af None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:33.255650 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd593f7-9500-4be9-bc11-b63d67bdd3af None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:33.266133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-0dd593f7-9500-4be9-bc11-b63d67bdd3af None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 16:23:33.266133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:33.266133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:23:33.266133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:23:33.266133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:23:33.266133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:23:33.266133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:23:33.266133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:33.266133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 16:23:33.266133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:33.266133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:33.266133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:33.266133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:33.266133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:33.266133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:33.266133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:33.267323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:33.267323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:33.267323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:33.267323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:33.267323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:33.267323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:33.267323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:33.267323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:23:33.267323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:23:33.267323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:23:33.267323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:23:33.267323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:33.267323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:33.267323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:33.267323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:33.267323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:33.268347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:33.268347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:33.268347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:33.268347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:33.268347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:33.268347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:33.268347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:33.268347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:33.268347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:33.268347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:33.268347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:33.268347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:23:33.268347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:33.268347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 16:23:33.268347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:33.275080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 418/834] 10.222.0.74 () {56 vars in 1132 bytes} [Wed Jul 27 16:23:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:33.276072 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd593f7-9500-4be9-bc11-b63d67bdd3af None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:33.287289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873387d-0a90-4cb1-b8cf-a20f7985c371 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:33.332253 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c910778-2df4-46b8-ae5f-970af38298c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:33.333845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c910778-2df4-46b8-ae5f-970af38298c8 None admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:33.334450 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c910778-2df4-46b8-ae5f-970af38298c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:33.334940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c910778-2df4-46b8-ae5f-970af38298c8 None admin] PATH_INFO: `/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/bb45ded9f75e4fa2973b564f83bdfaf2/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:33.337980 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c910778-2df4-46b8-ae5f-970af38298c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d6bcdd09cf24a5c8003951db37020cf, user_id=bb45ded9f75e4fa2973b564f83bdfaf2, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:33.340286 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c910778-2df4-46b8-ae5f-970af38298c8 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:33.357456 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 417/835] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:23:33 2022] PUT /identity/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/bb45ded9f75e4fa2973b564f83bdfaf2/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:23:33.389714 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd593f7-9500-4be9-bc11-b63d67bdd3af None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:33.405247 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:33.430238 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090cbab-983b-4b8d-a7d6-e6aefb643546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:33.430874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090cbab-983b-4b8d-a7d6-e6aefb643546 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:33.431162 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090cbab-983b-4b8d-a7d6-e6aefb643546 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:33.431436 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090cbab-983b-4b8d-a7d6-e6aefb643546 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:33.432125 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090cbab-983b-4b8d-a7d6-e6aefb643546 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:33.432591 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090cbab-983b-4b8d-a7d6-e6aefb643546 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:33.434102 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090cbab-983b-4b8d-a7d6-e6aefb643546 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:33.440144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 419/836] 10.222.0.74 () {56 vars in 1126 bytes} [Wed Jul 27 16:23:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:33.446418 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c910778-2df4-46b8-ae5f-970af38298c8 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:33.541473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:33.560937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c910778-2df4-46b8-ae5f-970af38298c8 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:33.596027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b25799-94a7-4a36-bc07-35b9a73c5860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:33.596650 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b25799-94a7-4a36-bc07-35b9a73c5860 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:33.596948 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b25799-94a7-4a36-bc07-35b9a73c5860 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:33.597223 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b25799-94a7-4a36-bc07-35b9a73c5860 None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:33.597894 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b25799-94a7-4a36-bc07-35b9a73c5860 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:33.598195 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b25799-94a7-4a36-bc07-35b9a73c5860 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:33.599557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b25799-94a7-4a36-bc07-35b9a73c5860 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:33.605037 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 418/837] 10.222.0.74 () {56 vars in 1126 bytes} [Wed Jul 27 16:23:33 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:34.183228 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabfc9b-1c8b-4d01-b0d7-9acb1654c140 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:34.183840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabfc9b-1c8b-4d01-b0d7-9acb1654c140 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:34.184322 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabfc9b-1c8b-4d01-b0d7-9acb1654c140 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:34.186194 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 420/838] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:34.191402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54ed57-56af-4777-b08a-6fb274fa66ec None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:34.192235 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54ed57-56af-4777-b08a-6fb274fa66ec None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:34.192713 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54ed57-56af-4777-b08a-6fb274fa66ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:34.211644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-be54ed57-56af-4777-b08a-6fb274fa66ec None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:34.213205 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-be54ed57-56af-4777-b08a-6fb274fa66ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:34.281671 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 419/839] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:34.302712 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabfc9b-1c8b-4d01-b0d7-9acb1654c140 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:34.413123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabfc9b-1c8b-4d01-b0d7-9acb1654c140 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:34.437609 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6baba2b-3c63-46b8-b0e8-80b5aa5a4413 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:34.437894 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6baba2b-3c63-46b8-b0e8-80b5aa5a4413 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:34.438106 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6baba2b-3c63-46b8-b0e8-80b5aa5a4413 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:34.440684 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 420/840] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:23:34.444496 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1299e1a-a4b9-4495-a5b9-43d5f753e15f None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:34.444783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1299e1a-a4b9-4495-a5b9-43d5f753e15f None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:34.445114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1299e1a-a4b9-4495-a5b9-43d5f753e15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:34.450619 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44dba4-b66b-4e18-9b75-8d420511b77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:34.451871 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44dba4-b66b-4e18-9b75-8d420511b77f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:34.452333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44dba4-b66b-4e18-9b75-8d420511b77f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:34.452782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44dba4-b66b-4e18-9b75-8d420511b77f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:34.453643 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f44dba4-b66b-4e18-9b75-8d420511b77f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:34.455013 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f44dba4-b66b-4e18-9b75-8d420511b77f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:34.466494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-4f44dba4-b66b-4e18-9b75-8d420511b77f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 16:23:34.466494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:34.466494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:23:34.466494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:23:34.466494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:23:34.466494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:23:34.466494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:23:34.466494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:34.466494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 16:23:34.466494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:34.466494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:34.466494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:34.466494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:34.466494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:34.466494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:34.466494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:34.466494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:34.467480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:34.467480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:34.467480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:34.467480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:34.467480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:34.467480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:34.467480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:23:34.467480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:23:34.467480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:23:34.467480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:23:34.467480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:34.467480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:34.467480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:34.467480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:34.467480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:34.467480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:34.468686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:34.468686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:34.468686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:34.468686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:34.468686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:34.468686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:34.468686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:34.468686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:34.468686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:34.468686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:34.468686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:23:34.468686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:34.468686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 16:23:34.468686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:34.468686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 421/841] 10.222.0.74 () {56 vars in 1114 bytes} [Wed Jul 27 16:23:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:34.472042 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-d1299e1a-a4b9-4495-a5b9-43d5f753e15f None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:34.473502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44dba4-b66b-4e18-9b75-8d420511b77f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:34.473894 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d1299e1a-a4b9-4495-a5b9-43d5f753e15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:34.521167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:34.577046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 421/842] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 16:23:34.586183 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44dba4-b66b-4e18-9b75-8d420511b77f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:34.593476 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1299e1a-a4b9-4495-a5b9-43d5f753e15f None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:34.603910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:34.622949 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841dda5-e996-43ed-966c-3a54ebdf3325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:34.623691 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841dda5-e996-43ed-966c-3a54ebdf3325 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:34.623968 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841dda5-e996-43ed-966c-3a54ebdf3325 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:34.624243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841dda5-e996-43ed-966c-3a54ebdf3325 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:34.624947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841dda5-e996-43ed-966c-3a54ebdf3325 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:34.625513 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841dda5-e996-43ed-966c-3a54ebdf3325 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:34.626775 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841dda5-e996-43ed-966c-3a54ebdf3325 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:34.632059 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 422/843] 10.222.0.74 () {56 vars in 1114 bytes} [Wed Jul 27 16:23:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:34.637942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841dda5-e996-43ed-966c-3a54ebdf3325 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:34.701612 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1299e1a-a4b9-4495-a5b9-43d5f753e15f None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:34.739215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd36a0-e70f-4ec6-b35a-2b3b53c77e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:34.740234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd36a0-e70f-4ec6-b35a-2b3b53c77e69 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:34.740745 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd36a0-e70f-4ec6-b35a-2b3b53c77e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:34.741253 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd36a0-e70f-4ec6-b35a-2b3b53c77e69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:34.744750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841dda5-e996-43ed-966c-3a54ebdf3325 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:34.752961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd36a0-e70f-4ec6-b35a-2b3b53c77e69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:34.758211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-1dbd36a0-e70f-4ec6-b35a-2b3b53c77e69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:34.758211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:34.758211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:34.758211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:34.758211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:34.758211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:34.758211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:34.758211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:34.758211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:34.758211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:34.758211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:34.758211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:34.758211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:34.758211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:34.758211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:34.758211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:34.759114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:34.759114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:34.759114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:34.759114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:34.759114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:34.759114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:34.759114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:34.759114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:34.759114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:34.759114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:34.759114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:34.759114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:34.759114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:34.759114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:34.759114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:34.759114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:34.759978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:34.759978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:34.759978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:34.759978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:34.759978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:34.759978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:34.759978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:34.759978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:34.759978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:34.759978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:34.759978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:23:34.759978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:34.767225 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 422/844] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:23:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:23:34.768625 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd36a0-e70f-4ec6-b35a-2b3b53c77e69 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:34.777239 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:34.781452 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb0fa2-e84b-4bb1-81be-270d1f8c71f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:34.782023 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb0fa2-e84b-4bb1-81be-270d1f8c71f2 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:34.782307 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb0fa2-e84b-4bb1-81be-270d1f8c71f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:34.782583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb0fa2-e84b-4bb1-81be-270d1f8c71f2 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:34.783257 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb0fa2-e84b-4bb1-81be-270d1f8c71f2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:34.783560 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb0fa2-e84b-4bb1-81be-270d1f8c71f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:34.785182 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb0fa2-e84b-4bb1-81be-270d1f8c71f2 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:34.797209 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 423/845] 10.222.0.74 () {56 vars in 1114 bytes} [Wed Jul 27 16:23:34 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:34.852705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:34.893534 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd36a0-e70f-4ec6-b35a-2b3b53c77e69 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:34.937829 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f3549-706e-41e0-b0bb-0a006c61ab17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:34.938461 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f3549-706e-41e0-b0bb-0a006c61ab17 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:34.938740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f3549-706e-41e0-b0bb-0a006c61ab17 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:34.939014 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f3549-706e-41e0-b0bb-0a006c61ab17 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:34.939739 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f3549-706e-41e0-b0bb-0a006c61ab17 None admin] RBAC: Adding query filter params (name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:34.940042 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f3549-706e-41e0-b0bb-0a006c61ab17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:34.941473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f3549-706e-41e0-b0bb-0a006c61ab17 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:34.951756 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 423/846] 10.222.0.74 () {56 vars in 1103 bytes} [Wed Jul 27 16:23:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:34.958002 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb0fa2-e84b-4bb1-81be-270d1f8c71f2 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:35.071439 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb0fa2-e84b-4bb1-81be-270d1f8c71f2 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:35.107619 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0121d-04e2-493f-8c70-05b1f9f7f3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:35.108360 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0121d-04e2-493f-8c70-05b1f9f7f3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:35.108651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0121d-04e2-493f-8c70-05b1f9f7f3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:35.108952 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0121d-04e2-493f-8c70-05b1f9f7f3b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:35.120951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0121d-04e2-493f-8c70-05b1f9f7f3b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:35.122486 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0121d-04e2-493f-8c70-05b1f9f7f3b1 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:35.138285 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-3ce0121d-04e2-493f-8c70-05b1f9f7f3b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:35.138285 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:35.138285 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:35.138285 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:35.138285 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:35.138285 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:35.138285 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:35.138285 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:35.138285 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:35.138285 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:35.138285 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:35.138285 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:35.138285 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:35.138285 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:35.138285 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:35.139201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:35.139201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:35.139201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:35.139201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:35.139201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:35.139201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:35.139201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:35.139201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:35.139201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:35.139201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:35.139201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:35.139201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:35.139201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:35.139201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:35.139201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:35.139201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:35.139201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:35.140077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:35.140077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:35.140077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:35.140077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:35.144078 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 424/847] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:35.145570 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f3549-706e-41e0-b0bb-0a006c61ab17 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:35.256008 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f3549-706e-41e0-b0bb-0a006c61ab17 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:35.289979 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:35.294262 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389a11f-eac0-4d4e-9961-bdee6d63295d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:35.295137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389a11f-eac0-4d4e-9961-bdee6d63295d None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:35.295618 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389a11f-eac0-4d4e-9961-bdee6d63295d None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:35.296114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389a11f-eac0-4d4e-9961-bdee6d63295d None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:35.297057 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389a11f-eac0-4d4e-9961-bdee6d63295d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:35.297576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389a11f-eac0-4d4e-9961-bdee6d63295d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:35.299281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389a11f-eac0-4d4e-9961-bdee6d63295d None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:35.312987 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 424/848] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:35.320912 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0121d-04e2-493f-8c70-05b1f9f7f3b1 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:35.342814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:35.426996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0121d-04e2-493f-8c70-05b1f9f7f3b1 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:35.455265 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c2308-e1c7-4bf4-913b-ababbaf72d87 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:35.455711 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c2308-e1c7-4bf4-913b-ababbaf72d87 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:35.456124 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c2308-e1c7-4bf4-913b-ababbaf72d87 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:35.457533 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 425/849] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:35.466296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457cc30-0b39-489c-81da-fe7b24d1728c None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:35.466575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457cc30-0b39-489c-81da-fe7b24d1728c None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:35.466847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457cc30-0b39-489c-81da-fe7b24d1728c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:35.467467 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e662c-8790-4c28-b6d9-850eb5359564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:35.468682 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e662c-8790-4c28-b6d9-850eb5359564 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:35.469206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e662c-8790-4c28-b6d9-850eb5359564 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:35.469836 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e662c-8790-4c28-b6d9-850eb5359564 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:35.478365 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e662c-8790-4c28-b6d9-850eb5359564 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:35.480140 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e662c-8790-4c28-b6d9-850eb5359564 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:35.484959 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-d457cc30-0b39-489c-81da-fe7b24d1728c None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:35.486144 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d457cc30-0b39-489c-81da-fe7b24d1728c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:35.492695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-389e662c-8790-4c28-b6d9-850eb5359564 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 16:23:35.492695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:35.492695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:35.492695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:35.492695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:35.492695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:35.492695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:35.492695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:35.492695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:35.492695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:35.492695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:35.492695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:35.492695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:35.492695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:35.492695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:35.492695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:35.493761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:35.493761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:35.493761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:35.493761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:35.493761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:35.493761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:35.493761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:35.493761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:35.493761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:35.493761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:35.493761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:35.493761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:35.493761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:35.493761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:35.493761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:35.493761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:35.493761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:35.493761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:35.494664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:35.494664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:35.494664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:35.494664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:35.494664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:35.494664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:35.494664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:35.494664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:35.494664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 16:23:35.494664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:35.497671 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 425/850] 10.222.0.74 () {56 vars in 1117 bytes} [Wed Jul 27 16:23:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:35.504779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e662c-8790-4c28-b6d9-850eb5359564 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:35.564103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 426/851] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 16:23:35.586286 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457cc30-0b39-489c-81da-fe7b24d1728c None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:35.596020 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:35.617967 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e662c-8790-4c28-b6d9-850eb5359564 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:35.654416 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bb40e-1799-495b-a846-90594fa8ab5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:35.655042 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bb40e-1799-495b-a846-90594fa8ab5c None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:35.655323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bb40e-1799-495b-a846-90594fa8ab5c None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:35.655593 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bb40e-1799-495b-a846-90594fa8ab5c None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:35.656381 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344bb40e-1799-495b-a846-90594fa8ab5c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:35.656686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344bb40e-1799-495b-a846-90594fa8ab5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:35.658128 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344bb40e-1799-495b-a846-90594fa8ab5c None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:35.675107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 426/852] 10.222.0.74 () {56 vars in 1149 bytes} [Wed Jul 27 16:23:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:35.676282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb094f3d-de3d-4ff0-a723-cc2d91c051b2 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:35.676565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb094f3d-de3d-4ff0-a723-cc2d91c051b2 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:35.676838 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb094f3d-de3d-4ff0-a723-cc2d91c051b2 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:35.681121 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 427/853] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:35.682028 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb094f3d-de3d-4ff0-a723-cc2d91c051b2 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:35.704882 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457cc30-0b39-489c-81da-fe7b24d1728c None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:35.725362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:35.740007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ed0a0-2385-4218-93c8-734eec11e7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:35.740649 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ed0a0-2385-4218-93c8-734eec11e7ee None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:35.740955 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ed0a0-2385-4218-93c8-734eec11e7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:35.741235 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ed0a0-2385-4218-93c8-734eec11e7ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:35.751845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ed0a0-2385-4218-93c8-734eec11e7ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:35.756864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-041ed0a0-2385-4218-93c8-734eec11e7ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:23:35.756864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:35.756864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:35.756864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:35.756864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:35.756864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:35.756864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:35.756864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:35.756864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:35.756864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:35.756864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:35.756864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:35.756864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:35.756864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:35.756864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:35.756864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:35.757778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:35.757778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:35.757778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:35.757778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:35.757778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:35.757778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:35.757778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:35.757778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:35.757778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:35.757778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:35.757778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:35.757778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:35.757778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:35.757778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:35.757778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:35.757778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:35.758603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:35.758603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:35.758603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:35.758603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:35.758603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:35.758603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:35.758603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:35.758603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:35.758603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:35.758603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:35.758603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:23:35.758603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:35.758603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 427/854] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:23:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:23:35.759278 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ebd8b-1564-4a50-943f-7587ea2cb237 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:35.759278 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ebd8b-1564-4a50-943f-7587ea2cb237 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:35.759905 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ebd8b-1564-4a50-943f-7587ea2cb237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:35.776858 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-f92ebd8b-1564-4a50-943f-7587ea2cb237 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:35.778112 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f92ebd8b-1564-4a50-943f-7587ea2cb237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:35.802445 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb094f3d-de3d-4ff0-a723-cc2d91c051b2 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:35.842270 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba052f-f1e4-49d0-bb8a-2be9fbef262f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:35.842973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba052f-f1e4-49d0-bb8a-2be9fbef262f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:35.843257 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba052f-f1e4-49d0-bb8a-2be9fbef262f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:35.843823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba052f-f1e4-49d0-bb8a-2be9fbef262f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:35.847957 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 428/855] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:35.848336 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ebd8b-1564-4a50-943f-7587ea2cb237 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:35.867005 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:35.932508 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba052f-f1e4-49d0-bb8a-2be9fbef262f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:35.934117 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba052f-f1e4-49d0-bb8a-2be9fbef262f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:35.954292 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ebd8b-1564-4a50-943f-7587ea2cb237 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:35.967543 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:35.993793 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37dcce-9977-485a-8e0d-11e7120e08b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:35.994419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37dcce-9977-485a-8e0d-11e7120e08b6 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:35.994699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37dcce-9977-485a-8e0d-11e7120e08b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:35.995502 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37dcce-9977-485a-8e0d-11e7120e08b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:35.996223 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37dcce-9977-485a-8e0d-11e7120e08b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:35.996531 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37dcce-9977-485a-8e0d-11e7120e08b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:35.997942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37dcce-9977-485a-8e0d-11e7120e08b6 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:36.007816 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 429/856] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:23:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:36.009737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37dcce-9977-485a-8e0d-11e7120e08b6 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:36.046272 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 428/857] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:23:35 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 16:23:36.047280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba052f-f1e4-49d0-bb8a-2be9fbef262f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:36.091502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:36.126020 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37dcce-9977-485a-8e0d-11e7120e08b6 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:36.156354 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba052f-f1e4-49d0-bb8a-2be9fbef262f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:36.167671 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0cd10-bc05-4460-a148-05dcec22753d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:36.168303 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0cd10-bc05-4460-a148-05dcec22753d None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:36.168970 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0cd10-bc05-4460-a148-05dcec22753d None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:36.168970 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0cd10-bc05-4460-a148-05dcec22753d None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:36.169552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e0cd10-bc05-4460-a148-05dcec22753d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:36.170993 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e0cd10-bc05-4460-a148-05dcec22753d None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:36.186957 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 430/858] 10.222.0.74 () {60 vars in 1141 bytes} [Wed Jul 27 16:23:36 2022] POST /identity/v3/services => generated 269 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:23:36.188068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0cd10-bc05-4460-a148-05dcec22753d None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:36.220263 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a7f48-f452-4de5-8203-40fc5ceb9695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:36.221277 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a7f48-f452-4de5-8203-40fc5ceb9695 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:36.221581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a7f48-f452-4de5-8203-40fc5ceb9695 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:36.223396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a7f48-f452-4de5-8203-40fc5ceb9695 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:36.235207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:36.248387 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a7f48-f452-4de5-8203-40fc5ceb9695 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:36.250938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a7f48-f452-4de5-8203-40fc5ceb9695 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:36.274282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-106a7f48-f452-4de5-8203-40fc5ceb9695 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:36.274282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:36.274282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:36.274282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:36.274282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:36.274282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:36.274282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:36.274282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:36.274282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:36.274282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:36.274282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:36.274282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:36.274282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:36.274282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:36.274282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:36.275231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:36.275231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:36.275231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:36.275231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:36.275231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:36.275231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:36.275231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:36.275231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:36.275231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:36.275231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:36.275231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:36.275231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:36.275231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:36.275231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:36.275231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:36.275231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:36.275231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:36.276638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:36.276638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:36.276638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:36.276638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:36.283002 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 429/859] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:36.286285 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a7f48-f452-4de5-8203-40fc5ceb9695 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:36.335633 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0cd10-bc05-4460-a148-05dcec22753d None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:36.387254 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c9a4b-4e0c-4ef3-8f96-56a3af9bc873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:36.388335 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c9a4b-4e0c-4ef3-8f96-56a3af9bc873 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:36.389155 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c9a4b-4e0c-4ef3-8f96-56a3af9bc873 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:36.389568 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:36.389826 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c9a4b-4e0c-4ef3-8f96-56a3af9bc873 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:36.405632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c9a4b-4e0c-4ef3-8f96-56a3af9bc873 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:36.407104 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c9a4b-4e0c-4ef3-8f96-56a3af9bc873 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:36.413865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a7f48-f452-4de5-8203-40fc5ceb9695 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:36.424668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-6a8c9a4b-4e0c-4ef3-8f96-56a3af9bc873 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:36.424668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:36.424668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:36.424668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:36.424668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:36.424668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:36.424668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:36.424668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:36.424668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:36.424668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:36.424668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:36.424668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:36.424668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:36.424668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:36.424668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:36.425644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:36.425644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:36.425644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:36.425644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:36.425644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:36.425644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:36.425644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:36.425644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:36.425644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:36.425644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:36.425644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:36.425644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:36.425644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:36.425644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:36.425644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:36.425644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:36.425644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:36.426735 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:36.426735 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:36.426735 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:36.426735 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:36.426735 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 431/860] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:36.431320 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c9a4b-4e0c-4ef3-8f96-56a3af9bc873 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:36.460987 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf3e3f-1bda-4a54-9087-c2c3efb504ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:36.462124 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf3e3f-1bda-4a54-9087-c2c3efb504ef None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:36.462580 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf3e3f-1bda-4a54-9087-c2c3efb504ef None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:36.463556 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf3e3f-1bda-4a54-9087-c2c3efb504ef None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:36.465623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf3e3f-1bda-4a54-9087-c2c3efb504ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:36.466246 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf3e3f-1bda-4a54-9087-c2c3efb504ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:36.469568 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf3e3f-1bda-4a54-9087-c2c3efb504ef None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:36.493487 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 430/861] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:36.495517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf3e3f-1bda-4a54-9087-c2c3efb504ef None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:36.544480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:36.564218 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c9a4b-4e0c-4ef3-8f96-56a3af9bc873 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:36.600448 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b255c7e-bbdc-40f4-afe8-3d636831c95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:36.600786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b255c7e-bbdc-40f4-afe8-3d636831c95f None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:36.601098 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b255c7e-bbdc-40f4-afe8-3d636831c95f None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:36.601430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b255c7e-bbdc-40f4-afe8-3d636831c95f None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:36.602120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b255c7e-bbdc-40f4-afe8-3d636831c95f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:36.602499 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b255c7e-bbdc-40f4-afe8-3d636831c95f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:36.603848 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b255c7e-bbdc-40f4-afe8-3d636831c95f None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:36.617002 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 432/862] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:36.622041 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b255c7e-bbdc-40f4-afe8-3d636831c95f None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:36.630944 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf3e3f-1bda-4a54-9087-c2c3efb504ef None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:36.669090 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc41da0-183a-44cb-b742-e55da10d488e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:36.669655 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc41da0-183a-44cb-b742-e55da10d488e None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:36.669940 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc41da0-183a-44cb-b742-e55da10d488e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:36.670219 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc41da0-183a-44cb-b742-e55da10d488e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:36.678186 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc41da0-183a-44cb-b742-e55da10d488e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:36.679406 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc41da0-183a-44cb-b742-e55da10d488e None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:36.691658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-ddc41da0-183a-44cb-b742-e55da10d488e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 16:23:36.691658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:36.691658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:36.691658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:36.691658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:36.691658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:36.691658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:36.691658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:36.691658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:36.691658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:36.691658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:36.691658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:36.691658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:36.691658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:36.691658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:36.691658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:36.692623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:36.692623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:36.692623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:36.692623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:36.692623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:36.692623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:36.692623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:36.692623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:36.692623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:36.692623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:36.692623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:36.692623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:36.692623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:36.692623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:36.692623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:36.692623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:36.692623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:36.692623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:36.693677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:36.693677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:36.693677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:36.693677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:36.693677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:36.693677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:36.693677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:36.693677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:36.693677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 16:23:36.693677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:36.693677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 431/863] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:23:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:23:36.701978 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc41da0-183a-44cb-b742-e55da10d488e None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:36.709366 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:36.745413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b255c7e-bbdc-40f4-afe8-3d636831c95f None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:36.783704 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0c9fa-b4bb-4e48-a236-bd9251345c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:36.784519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0c9fa-b4bb-4e48-a236-bd9251345c3b None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:36.784883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0c9fa-b4bb-4e48-a236-bd9251345c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:36.785303 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0c9fa-b4bb-4e48-a236-bd9251345c3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:36.793294 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f0c9fa-b4bb-4e48-a236-bd9251345c3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:36.794836 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f0c9fa-b4bb-4e48-a236-bd9251345c3b None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:36.806783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-f2f0c9fa-b4bb-4e48-a236-bd9251345c3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:36.806783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:36.806783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:36.806783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:36.806783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:36.806783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:36.806783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:36.806783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:36.806783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:36.806783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:36.806783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:36.806783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:36.806783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:36.806783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:36.806783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:36.808380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:36.808380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:36.808380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:36.808380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:36.808380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:36.808380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:36.808380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:36.808380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:36.808380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:36.808380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:36.808380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:36.808380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:36.808380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:36.808380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:36.808380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:36.808380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:36.808380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:36.809501 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:36.809501 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:36.809501 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:36.809501 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:36.809501 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:36.809501 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:36.809501 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:36.809501 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:36.810527 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 433/864] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:36.815928 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0c9fa-b4bb-4e48-a236-bd9251345c3b None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:36.821776 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:36.836083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc41da0-183a-44cb-b742-e55da10d488e None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:36.885392 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f3c96-1761-48e6-be1f-b1a0afde74c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:36.886907 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f3c96-1761-48e6-be1f-b1a0afde74c8 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:36.887679 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f3c96-1761-48e6-be1f-b1a0afde74c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:36.888352 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f3c96-1761-48e6-be1f-b1a0afde74c8 None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:36.889447 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f3c96-1761-48e6-be1f-b1a0afde74c8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:36.889903 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f3c96-1761-48e6-be1f-b1a0afde74c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:36.892080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f3c96-1761-48e6-be1f-b1a0afde74c8 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:36.914235 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 432/865] 10.222.0.74 () {56 vars in 1135 bytes} [Wed Jul 27 16:23:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:36.916093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e359601-64d0-45fd-9206-97139b122a65 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:36.916513 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e359601-64d0-45fd-9206-97139b122a65 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:36.917225 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e359601-64d0-45fd-9206-97139b122a65 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:36.918856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 433/866] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:36.920187 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e359601-64d0-45fd-9206-97139b122a65 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:36.932711 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0c9fa-b4bb-4e48-a236-bd9251345c3b None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:36.967636 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa6645-e8bd-49a7-a43c-09b1f6711aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:36.968249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa6645-e8bd-49a7-a43c-09b1f6711aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:36.968526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa6645-e8bd-49a7-a43c-09b1f6711aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:36.968797 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa6645-e8bd-49a7-a43c-09b1f6711aa8 None admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:36.969584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa6645-e8bd-49a7-a43c-09b1f6711aa8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:36.969885 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa6645-e8bd-49a7-a43c-09b1f6711aa8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:36.971306 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa6645-e8bd-49a7-a43c-09b1f6711aa8 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:36.988951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 434/867] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:36.988951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33577b11-6fa6-4f6d-8c5e-9f5cb7397520 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:36.988951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33577b11-6fa6-4f6d-8c5e-9f5cb7397520 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:36.988951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33577b11-6fa6-4f6d-8c5e-9f5cb7397520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:36.998237 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:37.005637 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-33577b11-6fa6-4f6d-8c5e-9f5cb7397520 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:37.006859 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-33577b11-6fa6-4f6d-8c5e-9f5cb7397520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:37.039087 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e359601-64d0-45fd-9206-97139b122a65 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:37.065576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:37.074447 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35feb4a7-2f0d-4a1e-bee6-24a3594de69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:37.075072 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35feb4a7-2f0d-4a1e-bee6-24a3594de69b None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:37.081248 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35feb4a7-2f0d-4a1e-bee6-24a3594de69b None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:37.081327 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 435/868] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:36 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:37.081427 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35feb4a7-2f0d-4a1e-bee6-24a3594de69b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:37.081502 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33577b11-6fa6-4f6d-8c5e-9f5cb7397520 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:37.155382 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35feb4a7-2f0d-4a1e-bee6-24a3594de69b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:37.156847 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35feb4a7-2f0d-4a1e-bee6-24a3594de69b None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:37.190470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33577b11-6fa6-4f6d-8c5e-9f5cb7397520 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:37.225279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:37.227469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388bb3d-ee62-492a-bf6f-a763697ebe5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:37.228126 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388bb3d-ee62-492a-bf6f-a763697ebe5b None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:37.228405 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388bb3d-ee62-492a-bf6f-a763697ebe5b None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:37.228703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388bb3d-ee62-492a-bf6f-a763697ebe5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:37.230181 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388bb3d-ee62-492a-bf6f-a763697ebe5b None admin] RBAC: Adding query filter params (role.id=2aa0cdfed849496db8f3692c9569d7ee, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=07d6bffc69674103b4c6540598a780b6) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:37.230422 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388bb3d-ee62-492a-bf6f-a763697ebe5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:37.231891 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388bb3d-ee62-492a-bf6f-a763697ebe5b None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:37.248925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 436/869] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:23:37 2022] GET /identity/v3/role_assignments?user.id=07d6bffc69674103b4c6540598a780b6&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=2aa0cdfed849496db8f3692c9569d7ee => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:37.248925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388bb3d-ee62-492a-bf6f-a763697ebe5b None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:37.286602 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 434/870] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:23:36 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 16:23:37.287722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35feb4a7-2f0d-4a1e-bee6-24a3594de69b None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:37.320139 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:37.387946 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388bb3d-ee62-492a-bf6f-a763697ebe5b None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:37.416072 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35feb4a7-2f0d-4a1e-bee6-24a3594de69b None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:37.436035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29241b72-5022-458c-a930-f38c2c20bbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:37.437035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29241b72-5022-458c-a930-f38c2c20bbd6 None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:37.437534 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29241b72-5022-458c-a930-f38c2c20bbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:37.438001 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29241b72-5022-458c-a930-f38c2c20bbd6 None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:37.438917 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29241b72-5022-458c-a930-f38c2c20bbd6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:37.440370 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29241b72-5022-458c-a930-f38c2c20bbd6 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:37.460608 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 437/871] 10.222.0.74 () {60 vars in 1141 bytes} [Wed Jul 27 16:23:37 2022] POST /identity/v3/services => generated 251 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:23:37.472513 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c65d87-3642-4e28-ac8f-1daecc476206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:37.473207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c65d87-3642-4e28-ac8f-1daecc476206 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:37.473487 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c65d87-3642-4e28-ac8f-1daecc476206 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:37.473758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c65d87-3642-4e28-ac8f-1daecc476206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:37.487950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c65d87-3642-4e28-ac8f-1daecc476206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:37.489832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c65d87-3642-4e28-ac8f-1daecc476206 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:37.508680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-f8c65d87-3642-4e28-ac8f-1daecc476206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:37.508680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:37.508680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:37.508680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:37.508680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:37.508680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:37.508680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:37.508680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:37.508680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:37.508680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:37.508680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:37.508680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:37.508680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:37.508680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:37.508680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:37.509938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:37.509938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:37.509938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:37.509938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:37.509938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:37.509938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:37.509938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:37.509938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:37.509938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:37.509938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:37.509938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:37.509938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:37.509938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:37.509938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:37.509938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:37.509938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:37.509938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:37.510911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:37.510911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:37.510911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:37.510911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:37.513818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 435/872] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:37.519169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29241b72-5022-458c-a930-f38c2c20bbd6 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:37.639957 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29241b72-5022-458c-a930-f38c2c20bbd6 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:37.658680 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:37.676960 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f5b35-9fe8-45a9-bcf7-7b6609339f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:37.677595 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f5b35-9fe8-45a9-bcf7-7b6609339f26 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:37.677882 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f5b35-9fe8-45a9-bcf7-7b6609339f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:37.678154 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f5b35-9fe8-45a9-bcf7-7b6609339f26 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:37.678851 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f5b35-9fe8-45a9-bcf7-7b6609339f26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:37.679155 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f5b35-9fe8-45a9-bcf7-7b6609339f26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:37.680526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f5b35-9fe8-45a9-bcf7-7b6609339f26 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:37.693636 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 438/873] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:37.699393 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c65d87-3642-4e28-ac8f-1daecc476206 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:37.715646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:37.810010 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c65d87-3642-4e28-ac8f-1daecc476206 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:37.846696 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98dc36-8c93-4e4a-a16a-45726268e467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:37.847689 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98dc36-8c93-4e4a-a16a-45726268e467 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:37.848165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98dc36-8c93-4e4a-a16a-45726268e467 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:37.848635 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98dc36-8c93-4e4a-a16a-45726268e467 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:37.856121 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce98dc36-8c93-4e4a-a16a-45726268e467 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:37.857855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce98dc36-8c93-4e4a-a16a-45726268e467 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:37.870216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-ce98dc36-8c93-4e4a-a16a-45726268e467 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:37.870216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:37.870216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:37.870216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:37.870216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:37.870216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:37.870216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:37.870216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:37.870216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:37.870216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:37.870216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:37.870216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:37.870216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:37.870216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:37.870216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:37.871089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:37.871089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:37.871089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:37.871089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:37.871089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:37.871089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:37.871089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:37.871089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:37.871089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:37.871089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:37.871089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:37.871089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:37.871089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:37.871089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:37.871089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:37.871089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:37.871089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:37.872037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:37.872037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:37.872037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:37.872037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:37.872037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:37.872037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:37.872037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:37.872037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:37.874450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 436/874] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:37.880942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f5b35-9fe8-45a9-bcf7-7b6609339f26 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:37.987948 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f5b35-9fe8-45a9-bcf7-7b6609339f26 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:38.023401 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8c5c5-8899-4512-9f1a-bbea8cb31390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:38.024016 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8c5c5-8899-4512-9f1a-bbea8cb31390 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:38.024290 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8c5c5-8899-4512-9f1a-bbea8cb31390 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:38.024560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8c5c5-8899-4512-9f1a-bbea8cb31390 None admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:38.025325 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8c5c5-8899-4512-9f1a-bbea8cb31390 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:38.025628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8c5c5-8899-4512-9f1a-bbea8cb31390 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:38.027080 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8c5c5-8899-4512-9f1a-bbea8cb31390 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:38.039476 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 439/875] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:38.045933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98dc36-8c93-4e4a-a16a-45726268e467 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:38.122524 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:38.155484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98dc36-8c93-4e4a-a16a-45726268e467 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:38.193037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf26162-6f84-4833-a663-a1fb4b72dfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:38.193661 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf26162-6f84-4833-a663-a1fb4b72dfb1 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:38.193939 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf26162-6f84-4833-a663-a1fb4b72dfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:38.194214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf26162-6f84-4833-a663-a1fb4b72dfb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:38.194998 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf26162-6f84-4833-a663-a1fb4b72dfb1 None admin] RBAC: Adding query filter params (role.id=ca37e43082fa40c6a88b12680a0ca94b, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=bb45ded9f75e4fa2973b564f83bdfaf2) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:38.195305 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf26162-6f84-4833-a663-a1fb4b72dfb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:38.196721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf26162-6f84-4833-a663-a1fb4b72dfb1 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:38.203905 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 437/876] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:23:38 2022] GET /identity/v3/role_assignments?user.id=bb45ded9f75e4fa2973b564f83bdfaf2&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=ca37e43082fa40c6a88b12680a0ca94b => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:38.277489 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329462d2-84e2-440a-b5e0-c55b3482189c None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:38.277694 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329462d2-84e2-440a-b5e0-c55b3482189c None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:38.277965 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329462d2-84e2-440a-b5e0-c55b3482189c None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:38.279692 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 440/877] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:38.285385 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4468667-83ea-456d-abbe-257bdf9368d8 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:38.285698 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4468667-83ea-456d-abbe-257bdf9368d8 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:38.285956 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4468667-83ea-456d-abbe-257bdf9368d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:38.305318 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-f4468667-83ea-456d-abbe-257bdf9368d8 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:38.306554 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-f4468667-83ea-456d-abbe-257bdf9368d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:38.384038 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 438/878] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:38.406446 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329462d2-84e2-440a-b5e0-c55b3482189c None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:38.448408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:38.510693 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329462d2-84e2-440a-b5e0-c55b3482189c None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:38.546591 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f359888-02e5-4789-a29f-260a11345435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:38.547168 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f359888-02e5-4789-a29f-260a11345435 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:38.547447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f359888-02e5-4789-a29f-260a11345435 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:38.547720 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f359888-02e5-4789-a29f-260a11345435 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:38.548353 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f359888-02e5-4789-a29f-260a11345435 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:38.549787 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f359888-02e5-4789-a29f-260a11345435 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:38.560249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-9f359888-02e5-4789-a29f-260a11345435 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 16:23:38.560249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:38.560249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:23:38.560249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:23:38.560249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:23:38.560249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:23:38.560249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:23:38.560249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:38.560249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 16:23:38.560249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:38.560249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:38.560249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:38.560249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:38.560249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:38.560249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:38.560249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:38.562482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:38.562482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:38.562482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:38.562482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:38.562482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:38.562482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:38.562482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:38.562482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:23:38.562482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:23:38.562482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:23:38.562482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:23:38.562482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:38.562482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:38.562482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:38.562482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:38.562482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:38.564462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:38.564462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:38.564462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:38.564462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:38.564462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:38.564462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:38.564462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:38.564462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:38.564462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:38.564462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:38.564462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:38.564462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:23:38.564462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:38.564462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 16:23:38.564462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:38.564462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 441/879] 10.222.0.74 () {56 vars in 1132 bytes} [Wed Jul 27 16:23:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:38.572915 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4468667-83ea-456d-abbe-257bdf9368d8 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:38.634502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:38.671211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4468667-83ea-456d-abbe-257bdf9368d8 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:38.707479 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce28a9-1ae3-4e3c-8e7d-b584a873273e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:38.708116 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce28a9-1ae3-4e3c-8e7d-b584a873273e None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:38.708395 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce28a9-1ae3-4e3c-8e7d-b584a873273e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:38.708666 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce28a9-1ae3-4e3c-8e7d-b584a873273e None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:38.709401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce28a9-1ae3-4e3c-8e7d-b584a873273e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:38.709705 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce28a9-1ae3-4e3c-8e7d-b584a873273e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:38.711087 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce28a9-1ae3-4e3c-8e7d-b584a873273e None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:38.716735 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 439/880] 10.222.0.74 () {56 vars in 1126 bytes} [Wed Jul 27 16:23:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:38.722419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f359888-02e5-4789-a29f-260a11345435 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:38.828248 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f359888-02e5-4789-a29f-260a11345435 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:38.829725 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:38.865803 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef33b34-e240-4feb-abb2-a59ae1a1507a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:38.866462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef33b34-e240-4feb-abb2-a59ae1a1507a None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:38.866751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef33b34-e240-4feb-abb2-a59ae1a1507a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:38.867026 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef33b34-e240-4feb-abb2-a59ae1a1507a None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:38.867692 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef33b34-e240-4feb-abb2-a59ae1a1507a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:38.867994 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef33b34-e240-4feb-abb2-a59ae1a1507a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:38.869443 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef33b34-e240-4feb-abb2-a59ae1a1507a None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:38.874790 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 442/881] 10.222.0.74 () {56 vars in 1126 bytes} [Wed Jul 27 16:23:38 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:38.880428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce28a9-1ae3-4e3c-8e7d-b584a873273e None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:38.990675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce28a9-1ae3-4e3c-8e7d-b584a873273e None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:39.024578 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:39.027840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e5398-70f3-4bd7-9a72-b80bc133c43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:39.028571 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e5398-70f3-4bd7-9a72-b80bc133c43f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:39.028858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e5398-70f3-4bd7-9a72-b80bc133c43f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:39.029164 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e5398-70f3-4bd7-9a72-b80bc133c43f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:39.030050 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e5398-70f3-4bd7-9a72-b80bc133c43f None admin] RBAC: Adding query filter params (interface=public, service_id=a89083dce83e4dda8bd3cf0072d97df2, region_id=RegionOne) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:39.030351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e5398-70f3-4bd7-9a72-b80bc133c43f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:39.031729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e5398-70f3-4bd7-9a72-b80bc133c43f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:39.040128 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 440/882] 10.222.0.74 () {56 vars in 1251 bytes} [Wed Jul 27 16:23:38 2022] GET /identity/v3/endpoints?service_id=a89083dce83e4dda8bd3cf0072d97df2&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:39.046011 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef33b34-e240-4feb-abb2-a59ae1a1507a None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:39.160318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef33b34-e240-4feb-abb2-a59ae1a1507a None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:39.200281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb453b-0c46-4474-9429-7e2dcf0b4d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:39.201194 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb453b-0c46-4474-9429-7e2dcf0b4d5a None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:39.201489 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb453b-0c46-4474-9429-7e2dcf0b4d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:39.201773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb453b-0c46-4474-9429-7e2dcf0b4d5a None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:39.202409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb453b-0c46-4474-9429-7e2dcf0b4d5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:39.203783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb453b-0c46-4474-9429-7e2dcf0b4d5a None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:39.210244 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 443/883] 10.222.0.74 () {56 vars in 1087 bytes} [Wed Jul 27 16:23:39 2022] GET /identity/v3/services => generated 1268 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:23:39.484844 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073986f9-bec3-464e-8fd5-d40115d7c849 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:39.485610 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073986f9-bec3-464e-8fd5-d40115d7c849 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:39.486108 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073986f9-bec3-464e-8fd5-d40115d7c849 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:39.487965 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 441/884] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:39.493771 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d9e3c-1616-4213-88b5-f227092c567d None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:39.494678 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d9e3c-1616-4213-88b5-f227092c567d None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:39.495709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d9e3c-1616-4213-88b5-f227092c567d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:39.514163 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-5d2d9e3c-1616-4213-88b5-f227092c567d None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:39.515652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-5d2d9e3c-1616-4213-88b5-f227092c567d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:39.541393 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:39.597901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 444/885] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:39.621129 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073986f9-bec3-464e-8fd5-d40115d7c849 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:39.624924 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb59908-d5cf-4c88-a6ad-312203ef770e None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:39.626154 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb59908-d5cf-4c88-a6ad-312203ef770e None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:39.626440 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb59908-d5cf-4c88-a6ad-312203ef770e None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:39.628036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 445/886] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:23:39.633652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d31781-f062-457a-adc9-7b714901da71 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:39.633940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d31781-f062-457a-adc9-7b714901da71 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:39.634217 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d31781-f062-457a-adc9-7b714901da71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:39.654594 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-e6d31781-f062-457a-adc9-7b714901da71 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:39.656099 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e6d31781-f062-457a-adc9-7b714901da71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:39.728352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 446/887] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 16:23:39.741086 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073986f9-bec3-464e-8fd5-d40115d7c849 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:39.750541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d31781-f062-457a-adc9-7b714901da71 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:39.785637 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68afe1b-842f-494e-802a-cb24e627c852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:39.786661 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68afe1b-842f-494e-802a-cb24e627c852 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:39.787422 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68afe1b-842f-494e-802a-cb24e627c852 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:39.787924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68afe1b-842f-494e-802a-cb24e627c852 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:39.802960 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68afe1b-842f-494e-802a-cb24e627c852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:39.810213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-c68afe1b-842f-494e-802a-cb24e627c852 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 16:23:39.810213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:39.810213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:39.810213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:39.810213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:39.810213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:39.810213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:39.810213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:39.810213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:39.810213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:39.810213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:39.810213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:39.810213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:39.810213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:39.810213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:39.810213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:39.811160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:39.811160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:39.811160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:39.811160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:39.811160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:39.811160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:39.811160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:39.811160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:39.811160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:39.811160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:39.811160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:39.811160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:39.811160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:39.811160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:39.811160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:39.811160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:39.812012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:39.812012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:39.812012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:39.812012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:39.812012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:39.812012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:39.812012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:39.812012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:39.812012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:39.812012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:39.812012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 16:23:39.812012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:39.815121 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 442/888] 10.222.0.74 () {56 vars in 1120 bytes} [Wed Jul 27 16:23:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:39.820416 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68afe1b-842f-494e-802a-cb24e627c852 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:39.855148 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:39.866506 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d31781-f062-457a-adc9-7b714901da71 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:39.887667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:39.908322 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad0da2-f496-4a14-8c5d-f485bfde4dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:39.909403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad0da2-f496-4a14-8c5d-f485bfde4dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:39.909904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad0da2-f496-4a14-8c5d-f485bfde4dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:39.910496 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad0da2-f496-4a14-8c5d-f485bfde4dc0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:39.911315 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad0da2-f496-4a14-8c5d-f485bfde4dc0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:39.913003 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad0da2-f496-4a14-8c5d-f485bfde4dc0 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:39.922845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-d7ad0da2-f496-4a14-8c5d-f485bfde4dc0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 16:23:39.922845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:39.922845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:23:39.922845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:23:39.922845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:23:39.922845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:23:39.922845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:23:39.922845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:39.922845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 16:23:39.922845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:39.922845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:39.922845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:39.922845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:39.922845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:39.922845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:39.922845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:39.922845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:39.923822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:39.923822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:39.923822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:39.923822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:39.923822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:39.923822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:39.923822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:23:39.923822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:23:39.923822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:23:39.923822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:23:39.923822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:39.923822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:39.923822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:39.923822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:39.923822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:39.923822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:39.924643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:39.924643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:39.924643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:39.924643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:39.924643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:39.924643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:39.924643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:39.924643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:39.924643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:39.924643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:39.924643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:23:39.924643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:39.924643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 16:23:39.924643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:39.932814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 447/889] 10.222.0.74 () {56 vars in 1114 bytes} [Wed Jul 27 16:23:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:23:39.934074 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad0da2-f496-4a14-8c5d-f485bfde4dc0 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:39.936186 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68afe1b-842f-494e-802a-cb24e627c852 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:39.973764 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61997628-3487-4365-adef-acf110016714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:39.974282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61997628-3487-4365-adef-acf110016714 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:39.974569 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61997628-3487-4365-adef-acf110016714 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:39.974842 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61997628-3487-4365-adef-acf110016714 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:39.975557 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61997628-3487-4365-adef-acf110016714 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:39.975866 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61997628-3487-4365-adef-acf110016714 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:39.977275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61997628-3487-4365-adef-acf110016714 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:39.990942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 443/890] 10.222.0.74 () {56 vars in 1115 bytes} [Wed Jul 27 16:23:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:39.992044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61997628-3487-4365-adef-acf110016714 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:40.045773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad0da2-f496-4a14-8c5d-f485bfde4dc0 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:40.082552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ee7d6-6372-4fa6-8a93-c1891e8e807a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:40.083502 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ee7d6-6372-4fa6-8a93-c1891e8e807a None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:40.084116 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ee7d6-6372-4fa6-8a93-c1891e8e807a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:40.084686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ee7d6-6372-4fa6-8a93-c1891e8e807a None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:40.085595 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ee7d6-6372-4fa6-8a93-c1891e8e807a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:40.086118 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ee7d6-6372-4fa6-8a93-c1891e8e807a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:40.087780 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ee7d6-6372-4fa6-8a93-c1891e8e807a None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:40.098565 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 448/891] 10.222.0.74 () {56 vars in 1114 bytes} [Wed Jul 27 16:23:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:40.099672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ee7d6-6372-4fa6-8a93-c1891e8e807a None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:40.100646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61997628-3487-4365-adef-acf110016714 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:40.118017 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:40.121972 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:40.141280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5e674-579e-4608-a6dc-3b40446a4c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:40.141937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5e674-579e-4608-a6dc-3b40446a4c57 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:40.142221 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5e674-579e-4608-a6dc-3b40446a4c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:40.142497 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5e674-579e-4608-a6dc-3b40446a4c57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:40.157138 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5e674-579e-4608-a6dc-3b40446a4c57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:40.158592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5e674-579e-4608-a6dc-3b40446a4c57 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:40.179942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-42b5e674-579e-4608-a6dc-3b40446a4c57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:40.179942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:40.179942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:40.179942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:40.179942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:40.179942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:40.179942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:40.179942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:40.179942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:40.179942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:40.179942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:40.179942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:40.179942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:40.179942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:40.179942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:40.180825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:40.180825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:40.180825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:40.180825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:40.180825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:40.180825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:40.180825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:40.180825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:40.180825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:40.180825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:40.180825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:40.180825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:40.180825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:40.180825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:40.180825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:40.180825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:40.180825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:40.181727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:40.181727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:40.181727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:40.181727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:40.186701 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 444/892] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:40.187771 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5e674-579e-4608-a6dc-3b40446a4c57 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:40.218281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ee7d6-6372-4fa6-8a93-c1891e8e807a None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:40.255831 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6058b-39d0-43a0-9a95-2df7bb5089a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:40.256853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6058b-39d0-43a0-9a95-2df7bb5089a6 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:40.257558 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6058b-39d0-43a0-9a95-2df7bb5089a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:40.258317 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6058b-39d0-43a0-9a95-2df7bb5089a6 None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:40.259298 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6058b-39d0-43a0-9a95-2df7bb5089a6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:40.259930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6058b-39d0-43a0-9a95-2df7bb5089a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:40.261687 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6058b-39d0-43a0-9a95-2df7bb5089a6 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:40.272458 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 449/893] 10.222.0.74 () {56 vars in 1114 bytes} [Wed Jul 27 16:23:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:40.273693 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6058b-39d0-43a0-9a95-2df7bb5089a6 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:40.295732 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5e674-579e-4608-a6dc-3b40446a4c57 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:40.332493 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c4d3af-8432-4b52-a33a-fc9e676cb7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:40.333147 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4d3af-8432-4b52-a33a-fc9e676cb7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:40.333446 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4d3af-8432-4b52-a33a-fc9e676cb7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:40.333707 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4d3af-8432-4b52-a33a-fc9e676cb7c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:40.334399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c4d3af-8432-4b52-a33a-fc9e676cb7c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:40.334714 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c4d3af-8432-4b52-a33a-fc9e676cb7c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:40.336088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c4d3af-8432-4b52-a33a-fc9e676cb7c3 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:40.349889 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 445/894] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:40.352192 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cdc42-ce2b-416e-ae38-834234ebdbf2 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:40.352478 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cdc42-ce2b-416e-ae38-834234ebdbf2 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:40.352748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cdc42-ce2b-416e-ae38-834234ebdbf2 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:40.359112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 446/895] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:40.359112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792cdc42-ce2b-416e-ae38-834234ebdbf2 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:40.368859 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:40.389251 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:40.390165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6058b-39d0-43a0-9a95-2df7bb5089a6 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:40.425635 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0048c6e-5b73-406c-9f98-46b043daab3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:40.426258 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0048c6e-5b73-406c-9f98-46b043daab3b None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:40.426540 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0048c6e-5b73-406c-9f98-46b043daab3b None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:40.426815 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0048c6e-5b73-406c-9f98-46b043daab3b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:40.427559 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0048c6e-5b73-406c-9f98-46b043daab3b None admin] RBAC: Adding query filter params (interface=public, service_id=5d0606fb6cfb4e39ab8793d7b410b180, region_id=RegionOne) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:40.427871 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0048c6e-5b73-406c-9f98-46b043daab3b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:40.429287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0048c6e-5b73-406c-9f98-46b043daab3b None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:40.435100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 450/896] 10.222.0.74 () {56 vars in 1251 bytes} [Wed Jul 27 16:23:40 2022] GET /identity/v3/endpoints?service_id=5d0606fb6cfb4e39ab8793d7b410b180&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:40.436264 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c3fa7-cedb-4132-bccd-65a8f4a5b6cf None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:40.436545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c3fa7-cedb-4132-bccd-65a8f4a5b6cf None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:40.436819 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c3fa7-cedb-4132-bccd-65a8f4a5b6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:40.454413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-470c3fa7-cedb-4132-bccd-65a8f4a5b6cf None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:40.455593 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-470c3fa7-cedb-4132-bccd-65a8f4a5b6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:40.477771 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792cdc42-ce2b-416e-ae38-834234ebdbf2 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:40.515560 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a187ec-0096-4d48-a937-5bd4b19db680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:40.516294 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a187ec-0096-4d48-a937-5bd4b19db680 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:40.516574 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a187ec-0096-4d48-a937-5bd4b19db680 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:40.516917 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a187ec-0096-4d48-a937-5bd4b19db680 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:40.524931 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 451/897] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:40.525422 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c3fa7-cedb-4132-bccd-65a8f4a5b6cf None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:40.529754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a187ec-0096-4d48-a937-5bd4b19db680 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:40.531365 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a187ec-0096-4d48-a937-5bd4b19db680 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:40.545924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-b3a187ec-0096-4d48-a937-5bd4b19db680 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 16:23:40.545924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:40.545924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:40.545924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:40.545924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:40.545924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:40.545924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:40.545924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:40.545924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:40.545924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:40.545924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:40.545924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:40.545924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:40.545924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:40.545924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:40.545924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:40.546942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:40.546942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:40.546942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:40.546942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:40.546942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:40.546942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:40.546942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:40.546942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:40.546942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:40.546942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:40.546942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:40.546942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:40.546942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:40.546942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:40.546942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:40.546942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:40.546942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:40.546942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:40.547867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:40.547867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:40.547867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:40.547867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:40.547867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:40.547867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:40.547867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:40.547867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:40.547867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 16:23:40.547867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:40.547867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 447/898] 10.222.0.74 () {56 vars in 1117 bytes} [Wed Jul 27 16:23:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:23:40.547867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a187ec-0096-4d48-a937-5bd4b19db680 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:40.622353 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:40.643992 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:40.647817 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c3fa7-cedb-4132-bccd-65a8f4a5b6cf None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:40.657964 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a187ec-0096-4d48-a937-5bd4b19db680 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:40.683503 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558a6f2-38f0-45fd-817b-e6a34b2d01ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:40.684117 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558a6f2-38f0-45fd-817b-e6a34b2d01ac None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:40.684399 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558a6f2-38f0-45fd-817b-e6a34b2d01ac None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:40.684667 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558a6f2-38f0-45fd-817b-e6a34b2d01ac None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:40.685348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558a6f2-38f0-45fd-817b-e6a34b2d01ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:40.686709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558a6f2-38f0-45fd-817b-e6a34b2d01ac None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:40.696931 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 452/899] 10.222.0.74 () {56 vars in 1087 bytes} [Wed Jul 27 16:23:40 2022] GET /identity/v3/services => generated 1268 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:23:40.697112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f47f2d-9c91-4697-875e-43ee3030cc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:40.697112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f47f2d-9c91-4697-875e-43ee3030cc7f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:40.697294 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558a6f2-38f0-45fd-817b-e6a34b2d01ac None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:40.697377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f47f2d-9c91-4697-875e-43ee3030cc7f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:40.697377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f47f2d-9c91-4697-875e-43ee3030cc7f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:40.697377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f47f2d-9c91-4697-875e-43ee3030cc7f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:40.697377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f47f2d-9c91-4697-875e-43ee3030cc7f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:40.715883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-53f47f2d-9c91-4697-875e-43ee3030cc7f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 16:23:40.715883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:40.715883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:23:40.715883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:23:40.715883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:23:40.715883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:23:40.715883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:23:40.715883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:40.715883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 16:23:40.715883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:40.715883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:40.715883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:40.715883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:40.715883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:40.715883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:40.715883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:40.715883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:40.717166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:40.717166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:40.717166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:40.717166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:40.717166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:40.717166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:40.717166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:23:40.717166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:23:40.717166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:23:40.717166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:23:40.717166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:40.717166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:40.717166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:40.717166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:40.717166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:40.717166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:40.718057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:40.718057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:40.718057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:40.718057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:40.718057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:40.718057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:40.718057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:40.718057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:40.718057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:40.718057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:40.718057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:23:40.718057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:40.718057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 16:23:40.718057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:40.718057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 448/900] 10.222.0.74 () {56 vars in 1126 bytes} [Wed Jul 27 16:23:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:40.723569 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f47f2d-9c91-4697-875e-43ee3030cc7f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:40.845006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558a6f2-38f0-45fd-817b-e6a34b2d01ac None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:40.873737 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f47f2d-9c91-4697-875e-43ee3030cc7f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:40.891971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce246eff-ca89-4e6c-ba4a-433e699ed257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:40.893148 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce246eff-ca89-4e6c-ba4a-433e699ed257 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:40.893148 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce246eff-ca89-4e6c-ba4a-433e699ed257 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:40.893148 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce246eff-ca89-4e6c-ba4a-433e699ed257 None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:40.893948 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce246eff-ca89-4e6c-ba4a-433e699ed257 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:40.894261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce246eff-ca89-4e6c-ba4a-433e699ed257 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:40.895831 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce246eff-ca89-4e6c-ba4a-433e699ed257 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:40.907036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:40.919578 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 453/901] 10.222.0.74 () {56 vars in 1149 bytes} [Wed Jul 27 16:23:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:40.925392 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce246eff-ca89-4e6c-ba4a-433e699ed257 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:40.927215 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75486b3f-2983-46df-8560-18986835d339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:40.927845 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75486b3f-2983-46df-8560-18986835d339 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:40.928305 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75486b3f-2983-46df-8560-18986835d339 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:40.928584 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75486b3f-2983-46df-8560-18986835d339 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:40.930149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75486b3f-2983-46df-8560-18986835d339 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:40.930458 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75486b3f-2983-46df-8560-18986835d339 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:40.931844 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75486b3f-2983-46df-8560-18986835d339 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:40.938605 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 449/902] 10.222.0.74 () {56 vars in 1122 bytes} [Wed Jul 27 16:23:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:40.944472 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75486b3f-2983-46df-8560-18986835d339 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:40.963370 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:41.038126 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce246eff-ca89-4e6c-ba4a-433e699ed257 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:41.062694 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75486b3f-2983-46df-8560-18986835d339 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:41.076973 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e30a21-431f-437a-873a-43946268b3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:41.077595 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e30a21-431f-437a-873a-43946268b3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:41.077875 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e30a21-431f-437a-873a-43946268b3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:41.078146 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e30a21-431f-437a-873a-43946268b3d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:41.107968 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeef014-82d3-4864-a335-ffcb11e56fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:41.108780 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeef014-82d3-4864-a335-ffcb11e56fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:41.109090 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeef014-82d3-4864-a335-ffcb11e56fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:41.109368 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeef014-82d3-4864-a335-ffcb11e56fe4 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:41.110318 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeef014-82d3-4864-a335-ffcb11e56fe4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:41.111047 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeef014-82d3-4864-a335-ffcb11e56fe4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:41.112676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeef014-82d3-4864-a335-ffcb11e56fe4 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:41.119507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 450/903] 10.222.0.74 () {56 vars in 1122 bytes} [Wed Jul 27 16:23:40 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:41.179515 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:41.185618 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e30a21-431f-437a-873a-43946268b3d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:41.187137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e30a21-431f-437a-873a-43946268b3d2 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:41.289210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feee140-b2f6-49bf-ad53-d1113dbcb3f4 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:41.290094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feee140-b2f6-49bf-ad53-d1113dbcb3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:41.290094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feee140-b2f6-49bf-ad53-d1113dbcb3f4 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:41.291277 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 451/904] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:23:41.297587 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80d558-e173-40dc-bd3e-bd00e6b9c741 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:41.298281 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80d558-e173-40dc-bd3e-bd00e6b9c741 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:41.298661 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80d558-e173-40dc-bd3e-bd00e6b9c741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:41.308128 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 454/905] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:23:40 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 16:23:41.315289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e30a21-431f-437a-873a-43946268b3d2 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:41.329224 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-9f80d558-e173-40dc-bd3e-bd00e6b9c741 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:41.330513 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9f80d558-e173-40dc-bd3e-bd00e6b9c741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:41.413020 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 452/906] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 16:23:41.438054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80d558-e173-40dc-bd3e-bd00e6b9c741 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:41.440548 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:41.443937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e30a21-431f-437a-873a-43946268b3d2 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:41.452475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:41.487111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43882de2-b724-4d53-8bf1-a3de6c18e8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:41.487766 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43882de2-b724-4d53-8bf1-a3de6c18e8de None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:41.488051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43882de2-b724-4d53-8bf1-a3de6c18e8de None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:41.488330 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43882de2-b724-4d53-8bf1-a3de6c18e8de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:41.503406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43882de2-b724-4d53-8bf1-a3de6c18e8de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:41.505620 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43882de2-b724-4d53-8bf1-a3de6c18e8de None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:41.525430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-43882de2-b724-4d53-8bf1-a3de6c18e8de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:41.525430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:41.525430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:41.525430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:41.525430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:41.525430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:41.525430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:41.525430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:41.525430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:41.525430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:41.525430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:41.525430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:41.525430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:41.525430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:41.525430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:41.527285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:41.527285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:41.527285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:41.527285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:41.527285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:41.527285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:41.527285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:41.527285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:41.527285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:41.527285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:41.527285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:41.527285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:41.527285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:41.527285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:41.527285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:41.527285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:41.527285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:41.528140 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:41.528140 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:41.528140 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:41.528140 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:41.534955 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 455/907] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:41.536061 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43882de2-b724-4d53-8bf1-a3de6c18e8de None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:41.560832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80d558-e173-40dc-bd3e-bd00e6b9c741 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:41.605565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de10802-bcd6-4e71-a13d-f5ab5a28ef3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:41.606388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de10802-bcd6-4e71-a13d-f5ab5a28ef3b None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:41.606687 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de10802-bcd6-4e71-a13d-f5ab5a28ef3b None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:41.606965 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de10802-bcd6-4e71-a13d-f5ab5a28ef3b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:41.607600 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de10802-bcd6-4e71-a13d-f5ab5a28ef3b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:41.609674 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de10802-bcd6-4e71-a13d-f5ab5a28ef3b None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:41.621338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-6de10802-bcd6-4e71-a13d-f5ab5a28ef3b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 16:23:41.621338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:41.621338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:23:41.621338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:23:41.621338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:23:41.621338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:23:41.621338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:23:41.621338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:41.621338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 16:23:41.621338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:41.621338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:41.621338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:41.621338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:41.621338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:41.621338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:41.621338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:41.622264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:41.622264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:41.622264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:41.622264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:41.622264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:41.622264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:41.622264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:41.622264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:23:41.622264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:23:41.622264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:23:41.622264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:23:41.622264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:41.622264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:41.622264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:41.622264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:41.622264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:41.623083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:41.623083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:41.623083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:41.623083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:41.623083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:41.623083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:41.623083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:41.623083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:41.623083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:41.623083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:41.623083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:41.623083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:23:41.623083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:41.623083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 16:23:41.623083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:41.624967 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 453/908] 10.222.0.74 () {56 vars in 1132 bytes} [Wed Jul 27 16:23:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:23:41.629824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de10802-bcd6-4e71-a13d-f5ab5a28ef3b None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:41.652199 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43882de2-b724-4d53-8bf1-a3de6c18e8de None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:41.688919 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad30d3b-b893-4ba2-b60b-87c16d19d0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:41.690015 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad30d3b-b893-4ba2-b60b-87c16d19d0db None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:41.690569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad30d3b-b893-4ba2-b60b-87c16d19d0db None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:41.691140 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad30d3b-b893-4ba2-b60b-87c16d19d0db None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:41.692022 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad30d3b-b893-4ba2-b60b-87c16d19d0db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:41.692519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad30d3b-b893-4ba2-b60b-87c16d19d0db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:41.694344 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad30d3b-b893-4ba2-b60b-87c16d19d0db None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:41.712522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 456/909] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:41.713779 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad30d3b-b893-4ba2-b60b-87c16d19d0db None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:41.721843 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:41.734647 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:41.753340 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de10802-bcd6-4e71-a13d-f5ab5a28ef3b None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:41.789597 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246bc04-2737-404b-ab0b-f604b82085b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:41.790578 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246bc04-2737-404b-ab0b-f604b82085b8 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:41.791071 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246bc04-2737-404b-ab0b-f604b82085b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:41.792297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246bc04-2737-404b-ab0b-f604b82085b8 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:41.793361 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b246bc04-2737-404b-ab0b-f604b82085b8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:41.793906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b246bc04-2737-404b-ab0b-f604b82085b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:41.795620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b246bc04-2737-404b-ab0b-f604b82085b8 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:41.801210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 454/910] 10.222.0.74 () {56 vars in 1126 bytes} [Wed Jul 27 16:23:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:41.807861 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246bc04-2737-404b-ab0b-f604b82085b8 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:41.844216 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad30d3b-b893-4ba2-b60b-87c16d19d0db None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:41.882218 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5d152-25ee-4ce3-a12c-5b74ce0fba9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:41.882596 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5d152-25ee-4ce3-a12c-5b74ce0fba9e None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:41.882877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5d152-25ee-4ce3-a12c-5b74ce0fba9e None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:41.883150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5d152-25ee-4ce3-a12c-5b74ce0fba9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:41.891310 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5d152-25ee-4ce3-a12c-5b74ce0fba9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:41.892763 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5d152-25ee-4ce3-a12c-5b74ce0fba9e None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:41.905207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-3ed5d152-25ee-4ce3-a12c-5b74ce0fba9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:41.905207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:41.905207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:41.905207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:41.905207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:41.905207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:41.905207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:41.905207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:41.905207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:41.905207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:41.905207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:41.905207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:41.905207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:41.905207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:41.905207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:41.906197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:41.906197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:41.906197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:41.906197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:41.906197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:41.906197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:41.906197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:41.906197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:41.906197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:41.906197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:41.906197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:41.906197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:41.906197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:41.906197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:41.906197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:41.906197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:41.906197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:41.907598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:41.907598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:41.907598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:41.907598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:41.907598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:41.907598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:41.907598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:41.907598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:41.907598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 457/911] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:41.917421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5d152-25ee-4ce3-a12c-5b74ce0fba9e None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:41.947579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246bc04-2737-404b-ab0b-f604b82085b8 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:41.997528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879272d6-99b5-43cc-88dd-4e5fa1039e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:41.998416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:42.000113 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879272d6-99b5-43cc-88dd-4e5fa1039e42 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:42.000679 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879272d6-99b5-43cc-88dd-4e5fa1039e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:42.001154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879272d6-99b5-43cc-88dd-4e5fa1039e42 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:42.001991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879272d6-99b5-43cc-88dd-4e5fa1039e42 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:42.002449 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879272d6-99b5-43cc-88dd-4e5fa1039e42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:42.003994 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879272d6-99b5-43cc-88dd-4e5fa1039e42 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:42.016991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 455/912] 10.222.0.74 () {56 vars in 1126 bytes} [Wed Jul 27 16:23:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:42.018714 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879272d6-99b5-43cc-88dd-4e5fa1039e42 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:42.022121 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:42.037761 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5d152-25ee-4ce3-a12c-5b74ce0fba9e None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:42.084680 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d9bd6-f06b-425c-bea0-70fa9b0764b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:42.085968 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d9bd6-f06b-425c-bea0-70fa9b0764b6 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:42.086676 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d9bd6-f06b-425c-bea0-70fa9b0764b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:42.087105 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d9bd6-f06b-425c-bea0-70fa9b0764b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:42.088257 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d9bd6-f06b-425c-bea0-70fa9b0764b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:42.089634 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d9bd6-f06b-425c-bea0-70fa9b0764b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:42.091218 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d9bd6-f06b-425c-bea0-70fa9b0764b6 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:42.109006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 458/913] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:42.115188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d9bd6-f06b-425c-bea0-70fa9b0764b6 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:42.145550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879272d6-99b5-43cc-88dd-4e5fa1039e42 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:42.187492 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa311ec8-6a4f-4df6-801d-d2f733bcd962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:42.187992 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa311ec8-6a4f-4df6-801d-d2f733bcd962 None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:42.188418 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa311ec8-6a4f-4df6-801d-d2f733bcd962 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:42.188790 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa311ec8-6a4f-4df6-801d-d2f733bcd962 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:42.189617 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa311ec8-6a4f-4df6-801d-d2f733bcd962 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:42.191181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa311ec8-6a4f-4df6-801d-d2f733bcd962 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:42.220753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 456/914] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:23:42 2022] POST /identity/v3/endpoints => generated 347 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:23:42.243764 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d9bd6-f06b-425c-bea0-70fa9b0764b6 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:42.282038 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:42.295544 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34f2a8-34ec-4371-93cc-e54c0362036a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:42.296171 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34f2a8-34ec-4371-93cc-e54c0362036a None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:42.296455 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34f2a8-34ec-4371-93cc-e54c0362036a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:42.296787 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34f2a8-34ec-4371-93cc-e54c0362036a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:42.297631 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34f2a8-34ec-4371-93cc-e54c0362036a None admin] RBAC: Adding query filter params (role.id=2bafbadedabe4d7681043ae2e7cadacd, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=07d6bffc69674103b4c6540598a780b6) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:42.297946 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34f2a8-34ec-4371-93cc-e54c0362036a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:42.299394 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34f2a8-34ec-4371-93cc-e54c0362036a None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:42.306253 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 459/915] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:23:42 2022] GET /identity/v3/role_assignments?user.id=07d6bffc69674103b4c6540598a780b6&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=2bafbadedabe4d7681043ae2e7cadacd => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:42.839559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32f7a3-f7b3-4f12-8832-6fc8c3dfb345 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:42.840095 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32f7a3-f7b3-4f12-8832-6fc8c3dfb345 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:42.840095 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32f7a3-f7b3-4f12-8832-6fc8c3dfb345 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:42.841620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 457/916] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:42.846797 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aac83b-0b17-4162-8a9a-7200cdf7db56 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:42.847613 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aac83b-0b17-4162-8a9a-7200cdf7db56 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:42.848092 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aac83b-0b17-4162-8a9a-7200cdf7db56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:42.866979 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-f8aac83b-0b17-4162-8a9a-7200cdf7db56 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:42.868507 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f8aac83b-0b17-4162-8a9a-7200cdf7db56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:42.940690 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 460/917] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:42 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:42.965046 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32f7a3-f7b3-4f12-8832-6fc8c3dfb345 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:43.025652 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:43.074096 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32f7a3-f7b3-4f12-8832-6fc8c3dfb345 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:43.108626 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd39a2-ae04-49d7-8bc4-a3c790446af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:43.109224 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd39a2-ae04-49d7-8bc4-a3c790446af4 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:43.109499 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd39a2-ae04-49d7-8bc4-a3c790446af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:43.109769 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd39a2-ae04-49d7-8bc4-a3c790446af4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:43.110446 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd39a2-ae04-49d7-8bc4-a3c790446af4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:43.111877 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd39a2-ae04-49d7-8bc4-a3c790446af4 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:43.121335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-b0fd39a2-ae04-49d7-8bc4-a3c790446af4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 16:23:43.121335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:43.121335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:23:43.121335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:23:43.121335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:23:43.121335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:23:43.121335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:23:43.121335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:43.121335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 16:23:43.121335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:43.121335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:43.121335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:43.121335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:43.121335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:43.121335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:43.121335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:43.121335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:43.122297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:43.122297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:43.122297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:43.122297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:43.122297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:43.122297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:43.122297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:23:43.122297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:23:43.122297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:23:43.122297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:23:43.122297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:43.122297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:43.122297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:43.122297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:43.122297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:43.122297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:43.123275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:43.123275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:43.123275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:43.123275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:43.123275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:43.123275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:43.123275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:43.123275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:43.123275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:43.123275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:43.123275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:23:43.123275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:43.123275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 16:23:43.123275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:43.123275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 458/918] 10.222.0.74 () {56 vars in 1114 bytes} [Wed Jul 27 16:23:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:43.128542 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aac83b-0b17-4162-8a9a-7200cdf7db56 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:43.183330 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb99f4f-c307-4785-8a18-e671b29b70b8 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:43.183615 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb99f4f-c307-4785-8a18-e671b29b70b8 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:43.183893 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb99f4f-c307-4785-8a18-e671b29b70b8 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:43.185384 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 459/919] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:23:43.190116 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d48b5-69a4-49e8-8d61-2aac847b6a9f None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:43.190396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d48b5-69a4-49e8-8d61-2aac847b6a9f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:43.190705 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d48b5-69a4-49e8-8d61-2aac847b6a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:43.209991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-9f2d48b5-69a4-49e8-8d61-2aac847b6a9f None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:43.211296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9f2d48b5-69a4-49e8-8d61-2aac847b6a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:43.233225 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aac83b-0b17-4162-8a9a-7200cdf7db56 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:43.255258 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:43.271321 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33914c-e95a-4285-9dc7-61bb97d21c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:43.272066 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33914c-e95a-4285-9dc7-61bb97d21c0c None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:43.272355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33914c-e95a-4285-9dc7-61bb97d21c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:43.272631 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33914c-e95a-4285-9dc7-61bb97d21c0c None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:43.273338 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33914c-e95a-4285-9dc7-61bb97d21c0c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:43.273642 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33914c-e95a-4285-9dc7-61bb97d21c0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:43.275130 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33914c-e95a-4285-9dc7-61bb97d21c0c None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:43.280590 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 461/920] 10.222.0.74 () {56 vars in 1114 bytes} [Wed Jul 27 16:23:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:43.285749 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33914c-e95a-4285-9dc7-61bb97d21c0c None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:43.293151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 460/921] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 16:23:43.312775 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d48b5-69a4-49e8-8d61-2aac847b6a9f None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:43.368421 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:43.389219 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33914c-e95a-4285-9dc7-61bb97d21c0c None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:43.427755 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d48b5-69a4-49e8-8d61-2aac847b6a9f None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:43.432062 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2825fc-158f-4105-9e5e-f139d47ab529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:43.432986 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2825fc-158f-4105-9e5e-f139d47ab529 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:43.433415 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2825fc-158f-4105-9e5e-f139d47ab529 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:43.433830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2825fc-158f-4105-9e5e-f139d47ab529 None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:43.434685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2825fc-158f-4105-9e5e-f139d47ab529 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:43.435193 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2825fc-158f-4105-9e5e-f139d47ab529 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:43.436786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2825fc-158f-4105-9e5e-f139d47ab529 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:43.446932 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 462/922] 10.222.0.74 () {56 vars in 1114 bytes} [Wed Jul 27 16:23:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:43.448630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2825fc-158f-4105-9e5e-f139d47ab529 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:43.471846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c30061-a2a3-473f-848d-113d8aad5aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:43.473105 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c30061-a2a3-473f-848d-113d8aad5aab None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:43.474028 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c30061-a2a3-473f-848d-113d8aad5aab None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:43.474469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c30061-a2a3-473f-848d-113d8aad5aab None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:43.475342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c30061-a2a3-473f-848d-113d8aad5aab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:43.477580 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c30061-a2a3-473f-848d-113d8aad5aab None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:43.504940 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 461/923] 10.222.0.74 () {60 vars in 1141 bytes} [Wed Jul 27 16:23:43 2022] POST /identity/v3/services => generated 241 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:23:43.515082 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:43.573987 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2825fc-158f-4105-9e5e-f139d47ab529 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:43.617579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef3e04-3e96-4890-a13d-12e60d7f673a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:43.618207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef3e04-3e96-4890-a13d-12e60d7f673a None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:43.618547 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef3e04-3e96-4890-a13d-12e60d7f673a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:43.618778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef3e04-3e96-4890-a13d-12e60d7f673a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:43.620061 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef3e04-3e96-4890-a13d-12e60d7f673a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:43.621509 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef3e04-3e96-4890-a13d-12e60d7f673a None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:43.645648 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 463/924] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:23:43 2022] POST /identity/v3/endpoints => generated 346 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:23:44.367645 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc970f-862d-42ba-b828-1a9b46b62295 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:44.368186 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc970f-862d-42ba-b828-1a9b46b62295 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:44.368186 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc970f-862d-42ba-b828-1a9b46b62295 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:44.370364 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 462/925] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:44.375534 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8965556-1d49-488c-a1a4-8e240d4af2ff None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:44.375920 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8965556-1d49-488c-a1a4-8e240d4af2ff None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:44.376101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8965556-1d49-488c-a1a4-8e240d4af2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:44.390088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80d583-c450-4561-94a2-c7ec6077da98 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:44.390377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80d583-c450-4561-94a2-c7ec6077da98 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:44.390953 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80d583-c450-4561-94a2-c7ec6077da98 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:44.392550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 463/926] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:23:44.394703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-d8965556-1d49-488c-a1a4-8e240d4af2ff None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:44.395899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d8965556-1d49-488c-a1a4-8e240d4af2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:44.397353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c5176f-6dd4-42e5-83e7-8157825ac0f4 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:44.397639 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c5176f-6dd4-42e5-83e7-8157825ac0f4 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:44.398120 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c5176f-6dd4-42e5-83e7-8157825ac0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:44.421001 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-b3c5176f-6dd4-42e5-83e7-8157825ac0f4 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:44.422727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b3c5176f-6dd4-42e5-83e7-8157825ac0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:44.476999 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 464/927] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:44.492616 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8965556-1d49-488c-a1a4-8e240d4af2ff None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:44.502252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 464/928] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 16:23:44.523757 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c5176f-6dd4-42e5-83e7-8157825ac0f4 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:44.526332 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:44.568435 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:44.601609 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8965556-1d49-488c-a1a4-8e240d4af2ff None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:44.629074 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c5176f-6dd4-42e5-83e7-8157825ac0f4 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:44.637379 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3782a-6578-4158-b3c5-32e48b0ca596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:44.638038 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3782a-6578-4158-b3c5-32e48b0ca596 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:44.638324 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3782a-6578-4158-b3c5-32e48b0ca596 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:44.638603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3782a-6578-4158-b3c5-32e48b0ca596 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:44.639236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3782a-6578-4158-b3c5-32e48b0ca596 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:44.640617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3782a-6578-4158-b3c5-32e48b0ca596 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:44.651083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-e7a3782a-6578-4158-b3c5-32e48b0ca596 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 16:23:44.651083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:44.651083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:23:44.651083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:23:44.651083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:23:44.651083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:23:44.651083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:23:44.651083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:44.651083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 16:23:44.651083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:44.651083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:44.651083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:44.651083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:44.651083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:44.651083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:44.651083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:44.651083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:44.652079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:44.652079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:44.652079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:44.652079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:44.652079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:44.652079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:44.652079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:23:44.652079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:23:44.652079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:23:44.652079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:23:44.652079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:44.652079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:44.652079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:44.652079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:44.652079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:44.652079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:44.652911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:44.652911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:44.652911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:44.652911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:44.652911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:44.652911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:44.652911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:44.652911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:44.652911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:44.652911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:44.652911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:23:44.652911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:44.652911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 16:23:44.652911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:44.657312 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 465/929] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:23:44.658499 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3782a-6578-4158-b3c5-32e48b0ca596 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:44.665954 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273603a-415b-4add-baf3-5f91b4402ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:44.667152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273603a-415b-4add-baf3-5f91b4402ced None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:44.667430 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273603a-415b-4add-baf3-5f91b4402ced None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:44.667706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273603a-415b-4add-baf3-5f91b4402ced None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:44.679934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273603a-415b-4add-baf3-5f91b4402ced None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:44.685514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-c273603a-415b-4add-baf3-5f91b4402ced None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 16:23:44.685514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:44.685514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:44.685514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:44.685514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:44.685514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:44.685514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:44.685514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:44.685514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:44.685514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:44.685514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:44.685514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:44.685514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:44.685514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:44.685514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:44.685514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:44.687676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:44.687676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:44.687676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:44.687676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:44.687676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:44.687676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:44.687676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:44.687676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:44.687676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:44.687676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:44.687676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:44.687676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:44.687676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:44.687676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:44.687676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:44.687676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:44.689651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:44.689651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:44.689651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:44.689651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:44.689651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:44.689651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:44.689651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:44.689651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:44.689651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:44.689651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:44.689651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 16:23:44.689651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:44.693052 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 465/930] 10.222.0.74 () {56 vars in 1120 bytes} [Wed Jul 27 16:23:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:23:44.693052 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273603a-415b-4add-baf3-5f91b4402ced None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:44.765615 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3782a-6578-4158-b3c5-32e48b0ca596 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:44.777239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:44.801490 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273603a-415b-4add-baf3-5f91b4402ced None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:44.803111 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:44.805820 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1317d-0de6-47d6-9dc4-25d8bd521068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:44.807609 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1317d-0de6-47d6-9dc4-25d8bd521068 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:44.807609 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1317d-0de6-47d6-9dc4-25d8bd521068 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:44.807609 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1317d-0de6-47d6-9dc4-25d8bd521068 None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:44.808215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1317d-0de6-47d6-9dc4-25d8bd521068 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:44.808523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1317d-0de6-47d6-9dc4-25d8bd521068 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:44.809981 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1317d-0de6-47d6-9dc4-25d8bd521068 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:44.820088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 466/931] 10.222.0.74 () {56 vars in 1112 bytes} [Wed Jul 27 16:23:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:44.821389 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1317d-0de6-47d6-9dc4-25d8bd521068 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:44.841274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222fb58-379f-443f-ad8c-33388416e939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:44.842147 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222fb58-379f-443f-ad8c-33388416e939 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:44.842574 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222fb58-379f-443f-ad8c-33388416e939 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:44.842991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222fb58-379f-443f-ad8c-33388416e939 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:44.843847 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222fb58-379f-443f-ad8c-33388416e939 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:44.844309 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222fb58-379f-443f-ad8c-33388416e939 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:44.845873 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222fb58-379f-443f-ad8c-33388416e939 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:44.859894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 466/932] 10.222.0.74 () {56 vars in 1115 bytes} [Wed Jul 27 16:23:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:44.860998 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222fb58-379f-443f-ad8c-33388416e939 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:44.925568 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1317d-0de6-47d6-9dc4-25d8bd521068 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:44.960201 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7d2f2-269b-4aec-a0ba-8b600921a15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:44.960843 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7d2f2-269b-4aec-a0ba-8b600921a15a None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:44.961145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7d2f2-269b-4aec-a0ba-8b600921a15a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:44.961422 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7d2f2-269b-4aec-a0ba-8b600921a15a None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:44.962106 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7d2f2-269b-4aec-a0ba-8b600921a15a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:44.962503 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7d2f2-269b-4aec-a0ba-8b600921a15a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:44.964232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7d2f2-269b-4aec-a0ba-8b600921a15a None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:44.966399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222fb58-379f-443f-ad8c-33388416e939 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:44.977061 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 467/933] 10.222.0.74 () {56 vars in 1112 bytes} [Wed Jul 27 16:23:44 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:45.008815 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4ab0b-a16e-44ae-aefe-deecd56aa9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:45.009795 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4ab0b-a16e-44ae-aefe-deecd56aa9ef None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:45.010269 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4ab0b-a16e-44ae-aefe-deecd56aa9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:45.010692 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4ab0b-a16e-44ae-aefe-deecd56aa9ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:45.028269 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4ab0b-a16e-44ae-aefe-deecd56aa9ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:45.029795 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4ab0b-a16e-44ae-aefe-deecd56aa9ef None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:45.046346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-fcd4ab0b-a16e-44ae-aefe-deecd56aa9ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:45.046346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:45.046346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:45.046346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:45.046346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:45.046346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:45.046346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:45.046346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:45.046346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:45.046346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:45.046346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:45.046346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:45.046346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:45.046346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:45.046346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:45.047291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:45.047291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:45.047291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:45.047291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:45.047291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:45.047291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:45.047291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:45.047291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:45.047291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:45.047291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:45.047291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:45.047291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:45.047291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:45.047291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:45.047291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:45.047291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:45.047291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:45.048269 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:45.048269 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:45.048269 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:45.048269 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:45.048269 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 467/934] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:45.054811 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7d2f2-269b-4aec-a0ba-8b600921a15a None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:45.076491 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:45.173818 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7d2f2-269b-4aec-a0ba-8b600921a15a None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:45.211134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f6ea0-934c-43f2-a126-453e2ff0d69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:45.212100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f6ea0-934c-43f2-a126-453e2ff0d69c None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:45.212582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f6ea0-934c-43f2-a126-453e2ff0d69c None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:45.213111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f6ea0-934c-43f2-a126-453e2ff0d69c None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:45.214559 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f6ea0-934c-43f2-a126-453e2ff0d69c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:45.215064 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f6ea0-934c-43f2-a126-453e2ff0d69c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:45.216626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f6ea0-934c-43f2-a126-453e2ff0d69c None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:45.230173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 468/935] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:45.235997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4ab0b-a16e-44ae-aefe-deecd56aa9ef None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:45.277357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:45.345612 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4ab0b-a16e-44ae-aefe-deecd56aa9ef None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:45.381296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe6653-9cdd-4ac1-9436-4451f07677c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:45.382764 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe6653-9cdd-4ac1-9436-4451f07677c8 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:45.383277 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe6653-9cdd-4ac1-9436-4451f07677c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:45.383751 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe6653-9cdd-4ac1-9436-4451f07677c8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:45.392752 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe6653-9cdd-4ac1-9436-4451f07677c8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:45.394508 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe6653-9cdd-4ac1-9436-4451f07677c8 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:45.406973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-f5fe6653-9cdd-4ac1-9436-4451f07677c8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 16:23:45.406973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:45.406973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:45.406973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:45.406973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:45.406973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:45.406973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:45.406973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:45.406973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:45.406973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:45.406973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:45.406973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:45.406973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:45.406973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:45.406973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:45.406973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:45.407887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:45.407887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:45.407887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:45.407887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:45.407887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:45.407887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:45.407887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:45.407887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:45.407887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:45.407887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:45.407887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:45.407887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:45.407887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:45.407887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:45.407887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:45.407887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:45.407887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:45.407887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:45.408806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:45.408806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:45.408806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:45.408806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:45.408806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:45.408806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:45.408806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:45.408806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:45.408806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 16:23:45.408806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:45.411285 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 468/936] 10.222.0.74 () {56 vars in 1117 bytes} [Wed Jul 27 16:23:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:45.417199 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f6ea0-934c-43f2-a126-453e2ff0d69c None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:45.502811 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:45.526348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f6ea0-934c-43f2-a126-453e2ff0d69c None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:45.563132 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11649be0-2924-4db4-8918-382e4a6907b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:45.564148 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11649be0-2924-4db4-8918-382e4a6907b5 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:45.564631 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11649be0-2924-4db4-8918-382e4a6907b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:45.565136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11649be0-2924-4db4-8918-382e4a6907b5 None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:45.566131 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11649be0-2924-4db4-8918-382e4a6907b5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:45.566636 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11649be0-2924-4db4-8918-382e4a6907b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:45.568284 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11649be0-2924-4db4-8918-382e4a6907b5 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:45.587594 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 469/937] 10.222.0.74 () {56 vars in 1149 bytes} [Wed Jul 27 16:23:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:45.594081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe6653-9cdd-4ac1-9436-4451f07677c8 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:45.608235 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79500feb-4ea7-4208-8bd6-3bb4459c632a None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:45.608550 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79500feb-4ea7-4208-8bd6-3bb4459c632a None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:45.608788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79500feb-4ea7-4208-8bd6-3bb4459c632a None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:45.613248 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 470/938] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:45.615650 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd726e89-6696-47ac-8bbc-30ab75ff51c7 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:45.615934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd726e89-6696-47ac-8bbc-30ab75ff51c7 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:45.616205 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd726e89-6696-47ac-8bbc-30ab75ff51c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:45.634893 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-bd726e89-6696-47ac-8bbc-30ab75ff51c7 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:45.636705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-bd726e89-6696-47ac-8bbc-30ab75ff51c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:45.695692 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:45.718906 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 471/939] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 16:23:45.720172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a41f31-963e-422a-a864-90c42299fddd None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:45.720458 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a41f31-963e-422a-a864-90c42299fddd None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:45.720732 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a41f31-963e-422a-a864-90c42299fddd None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:45.722151 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 472/940] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:45.727054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd14f18-b295-462e-b0a8-0e121b9ba019 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:45.727336 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd14f18-b295-462e-b0a8-0e121b9ba019 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:45.727607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd14f18-b295-462e-b0a8-0e121b9ba019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:45.732585 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe6653-9cdd-4ac1-9436-4451f07677c8 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:45.747209 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-fdd14f18-b295-462e-b0a8-0e121b9ba019 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:45.748380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-fdd14f18-b295-462e-b0a8-0e121b9ba019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:45.768199 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3b403-4a9d-4655-af88-33faa2bb0f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:45.769132 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3b403-4a9d-4655-af88-33faa2bb0f7a None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:45.769610 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3b403-4a9d-4655-af88-33faa2bb0f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:45.770066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3b403-4a9d-4655-af88-33faa2bb0f7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:45.780665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3b403-4a9d-4655-af88-33faa2bb0f7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:45.782142 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3b403-4a9d-4655-af88-33faa2bb0f7a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:45.796255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-6af3b403-4a9d-4655-af88-33faa2bb0f7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:45.796255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:45.796255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:45.796255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:45.796255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:45.796255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:45.796255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:45.796255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:45.796255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:45.796255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:45.796255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:45.796255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:45.796255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:45.796255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:45.796255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:45.797218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:45.797218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:45.797218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:45.797218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:45.797218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:45.797218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:45.797218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:45.797218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:45.797218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:45.797218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:45.797218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:45.797218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:45.797218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:45.797218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:45.797218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:45.797218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:45.797218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:45.802903 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:45.802903 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:45.802903 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:45.802903 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:45.802903 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 469/941] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:45.802903 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3b403-4a9d-4655-af88-33faa2bb0f7a None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:45.820104 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:45.832926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 473/942] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 16:23:45.832926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd14f18-b295-462e-b0a8-0e121b9ba019 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:45.916163 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3b403-4a9d-4655-af88-33faa2bb0f7a None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:45.935135 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd14f18-b295-462e-b0a8-0e121b9ba019 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:45.943936 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:45.953588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3390b-3c70-443f-a3cb-629744104a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:45.954283 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3390b-3c70-443f-a3cb-629744104a17 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:45.954561 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3390b-3c70-443f-a3cb-629744104a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:45.954853 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3390b-3c70-443f-a3cb-629744104a17 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:45.955476 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3390b-3c70-443f-a3cb-629744104a17 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:45.956842 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3390b-3c70-443f-a3cb-629744104a17 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:45.966214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-26c3390b-3c70-443f-a3cb-629744104a17 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 16:23:45.966214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:45.966214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:23:45.966214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:23:45.966214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:23:45.966214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:23:45.966214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:23:45.966214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:45.966214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 16:23:45.966214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:45.966214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:45.966214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:45.966214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:45.966214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:45.966214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:45.966214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:45.966214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:45.967170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:45.967170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:45.967170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:45.967170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:45.967170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:45.967170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:45.967170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:23:45.967170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:23:45.967170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:23:45.967170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:23:45.967170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:45.967170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:45.967170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:45.967170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:45.967170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:45.967170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:45.967991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:45.967991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:45.967991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:45.967991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:45.967991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:45.967991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:45.967991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:45.967991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:45.967991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:45.967991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:45.967991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:23:45.967991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:45.967991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 16:23:45.967991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:45.967991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 470/943] 10.222.0.74 () {56 vars in 1126 bytes} [Wed Jul 27 16:23:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:45.968769 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3390b-3c70-443f-a3cb-629744104a17 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:45.977676 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af61754-2ccc-4695-8fdd-f317a7da54eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:45.978516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af61754-2ccc-4695-8fdd-f317a7da54eb None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:45.979541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af61754-2ccc-4695-8fdd-f317a7da54eb None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:45.979991 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af61754-2ccc-4695-8fdd-f317a7da54eb None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:45.981015 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af61754-2ccc-4695-8fdd-f317a7da54eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:45.981389 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af61754-2ccc-4695-8fdd-f317a7da54eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:45.982852 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af61754-2ccc-4695-8fdd-f317a7da54eb None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:45.995370 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 474/944] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:45.996285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af61754-2ccc-4695-8fdd-f317a7da54eb None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:46.053349 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:46.081439 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3390b-3c70-443f-a3cb-629744104a17 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:46.101533 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af61754-2ccc-4695-8fdd-f317a7da54eb None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:46.115122 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d463bc-51ee-41a1-b1a4-4a5bad543bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:46.115738 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d463bc-51ee-41a1-b1a4-4a5bad543bda None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:46.116018 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d463bc-51ee-41a1-b1a4-4a5bad543bda None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:46.116289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d463bc-51ee-41a1-b1a4-4a5bad543bda None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:46.116980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d463bc-51ee-41a1-b1a4-4a5bad543bda None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:46.118438 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d463bc-51ee-41a1-b1a4-4a5bad543bda None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:46.121571 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: WARNING keystone.server.flask.common [None req-19d463bc-51ee-41a1-b1a4-4a5bad543bda None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 27 16:23:46.138421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6def275-3b36-4b40-ad9d-3a6a795d79bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:46.138943 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6def275-3b36-4b40-ad9d-3a6a795d79bf None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:46.139210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6def275-3b36-4b40-ad9d-3a6a795d79bf None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:46.139482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6def275-3b36-4b40-ad9d-3a6a795d79bf None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:46.140149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6def275-3b36-4b40-ad9d-3a6a795d79bf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:46.140452 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6def275-3b36-4b40-ad9d-3a6a795d79bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:46.140809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.sql.core [None req-19d463bc-51ee-41a1-b1a4-4a5bad543bda None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 16:23:46.140809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 27 16:23:46.140809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [parameters: {'id': 'b935b3dbdb944cdab98fd30616236c16', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 27 16:23:46.140809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80110) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 16:23:46.141842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6def275-3b36-4b40-ad9d-3a6a795d79bf None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:46.146723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 475/945] 10.222.0.74 () {56 vars in 1122 bytes} [Wed Jul 27 16:23:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:46.147530 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6def275-3b36-4b40-ad9d-3a6a795d79bf None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:46.158100 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-19d463bc-51ee-41a1-b1a4-4a5bad543bda None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b935b3dbdb944cdab98fd30616236c16.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b935b3dbdb944cdab98fd30616236c16. Jul 27 16:23:46.158100 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:46.158100 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 16:23:46.158100 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 16:23:46.158100 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 16:23:46.158100 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 16:23:46.158100 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 16:23:46.158100 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 16:23:46.158100 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 16:23:46.158100 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application conn.query(q) Jul 27 16:23:46.158100 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 16:23:46.158100 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 16:23:46.158100 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 16:23:46.159004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application result.read() Jul 27 16:23:46.159004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 16:23:46.159004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 16:23:46.159004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 16:23:46.159004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 16:23:46.159004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 16:23:46.159004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 16:23:46.159004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 16:23:46.159004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 16:23:46.159004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 16:23:46.159004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:46.159004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 16:23:46.159004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:46.159004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:46.159004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 16:23:46.159004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 16:23:46.159004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 27 16:23:46.159004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 27 16:23:46.159942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 16:23:46.159942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application next(self.gen) Jul 27 16:23:46.159942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 16:23:46.159942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application yield resource Jul 27 16:23:46.159942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 16:23:46.159942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application next(self.gen) Jul 27 16:23:46.159942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 16:23:46.159942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 16:23:46.159942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 16:23:46.159942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 16:23:46.159942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 16:23:46.159942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise self.value Jul 27 16:23:46.159942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 16:23:46.159942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 16:23:46.159942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 16:23:46.159942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application session.commit() Jul 27 16:23:46.159942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 16:23:46.160819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 16:23:46.160819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 16:23:46.160819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 16:23:46.160819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 16:23:46.160819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self.session.flush() Jul 27 16:23:46.160819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 16:23:46.160819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 16:23:46.160819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 16:23:46.160819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 16:23:46.160819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 16:23:46.160819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application compat.raise_( Jul 27 16:23:46.160819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 16:23:46.160819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception Jul 27 16:23:46.160819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 16:23:46.160819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 16:23:46.160819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 16:23:46.160819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 16:23:46.161779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 16:23:46.161779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 16:23:46.161779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 16:23:46.161779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 16:23:46.161779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 27 16:23:46.161779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 27 16:23:46.161779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 16:23:46.161779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 16:23:46.161779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 16:23:46.161779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 16:23:46.161779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 16:23:46.161779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 16:23:46.161779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 16:23:46.161779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 16:23:46.161779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 16:23:46.162565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 16:23:46.162565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 16:23:46.162565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception Jul 27 16:23:46.162565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 16:23:46.162565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 16:23:46.162565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 16:23:46.162565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 16:23:46.162565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 16:23:46.162565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 16:23:46.162565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 16:23:46.162565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application conn.query(q) Jul 27 16:23:46.162565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 16:23:46.162565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 16:23:46.162565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 16:23:46.162565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application result.read() Jul 27 16:23:46.162565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 16:23:46.162565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 16:23:46.163417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 16:23:46.163417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 16:23:46.163417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 16:23:46.163417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 16:23:46.163417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 16:23:46.163417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 16:23:46.163417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 16:23:46.163417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 27 16:23:46.163417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [parameters: {'id': 'b935b3dbdb944cdab98fd30616236c16', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 27 16:23:46.163417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 16:23:46.163417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:46.163417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:46.163417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:46.163417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:46.163417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 27 16:23:46.164177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 27 16:23:46.164177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 16:23:46.164177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 16:23:46.164177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 27 16:23:46.164177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:46.164177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:46.164177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:46.164177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:46.164177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:46.164177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:46.164177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:46.164177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:46.164177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:46.164177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:46.164177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:46.164177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:46.164177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:46.166145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:46.166145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 27 16:23:46.166145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 27 16:23:46.166145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:46.166145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:46.166145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 27 16:23:46.166145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 27 16:23:46.166145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b935b3dbdb944cdab98fd30616236c16. Jul 27 16:23:46.166145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:46.166145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 471/946] 10.222.0.74 () {60 vars in 1140 bytes} [Wed Jul 27 16:23:45 2022] POST /identity/v3/projects => generated 262 bytes in 192 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:46.166145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d463bc-51ee-41a1-b1a4-4a5bad543bda None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:46.218596 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:46.274910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6def275-3b36-4b40-ad9d-3a6a795d79bf None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:46.279283 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d463bc-51ee-41a1-b1a4-4a5bad543bda None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:46.295820 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:46.314715 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6832a3d-039c-47b3-b56e-e3f67170375f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:46.315293 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6832a3d-039c-47b3-b56e-e3f67170375f None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:46.315576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6832a3d-039c-47b3-b56e-e3f67170375f None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:46.315851 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6832a3d-039c-47b3-b56e-e3f67170375f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:46.317323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4cbd5-551d-480e-9e8d-0fb4da27de6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:46.318056 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4cbd5-551d-480e-9e8d-0fb4da27de6a None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:46.318374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4cbd5-551d-480e-9e8d-0fb4da27de6a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:46.318650 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4cbd5-551d-480e-9e8d-0fb4da27de6a None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:46.319325 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4cbd5-551d-480e-9e8d-0fb4da27de6a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:46.319628 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4cbd5-551d-480e-9e8d-0fb4da27de6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:46.321208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4cbd5-551d-480e-9e8d-0fb4da27de6a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:46.324202 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6832a3d-039c-47b3-b56e-e3f67170375f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:46.326162 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6832a3d-039c-47b3-b56e-e3f67170375f None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:46.329464 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 472/947] 10.222.0.74 () {56 vars in 1122 bytes} [Wed Jul 27 16:23:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:46.330232 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4cbd5-551d-480e-9e8d-0fb4da27de6a None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:46.340103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-f6832a3d-039c-47b3-b56e-e3f67170375f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:46.340103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:46.340103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:46.340103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:46.340103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:46.340103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:46.340103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:46.340103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:46.340103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:46.340103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:46.340103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:46.340103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:46.340103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:46.340103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:46.340103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:46.341036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:46.341036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:46.341036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:46.341036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:46.341036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:46.341036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:46.341036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:46.341036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:46.341036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:46.341036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:46.341036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:46.341036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:46.341036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:46.341036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:46.341036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:46.341036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:46.341036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:46.341902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:46.341902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:46.341902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:46.341902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:46.341902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:46.341902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:46.341902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:46.341902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:46.345152 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 476/948] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:46.345833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6832a3d-039c-47b3-b56e-e3f67170375f None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:46.442499 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4cbd5-551d-480e-9e8d-0fb4da27de6a None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:46.445149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6832a3d-039c-47b3-b56e-e3f67170375f None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:46.457182 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:46.480419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1debd9-17fb-4a7a-bac7-3c2a5952283a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:46.480640 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234e675-81d8-47ce-898a-012a63bfdb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:46.481230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234e675-81d8-47ce-898a-012a63bfdb64 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:46.481509 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234e675-81d8-47ce-898a-012a63bfdb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:46.481782 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234e675-81d8-47ce-898a-012a63bfdb64 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:46.481913 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1debd9-17fb-4a7a-bac7-3c2a5952283a None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:46.482336 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1debd9-17fb-4a7a-bac7-3c2a5952283a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:46.482522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234e675-81d8-47ce-898a-012a63bfdb64 None admin] RBAC: Adding query filter params (interface=public, service_id=9031214db68a40c0b596671be7b70f2c, region_id=RegionOne) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:46.482828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234e675-81d8-47ce-898a-012a63bfdb64 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:46.483044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1debd9-17fb-4a7a-bac7-3c2a5952283a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:46.484190 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234e675-81d8-47ce-898a-012a63bfdb64 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:46.489138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 477/949] 10.222.0.74 () {56 vars in 1251 bytes} [Wed Jul 27 16:23:46 2022] GET /identity/v3/endpoints?service_id=9031214db68a40c0b596671be7b70f2c&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:46.489909 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234e675-81d8-47ce-898a-012a63bfdb64 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:46.495161 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1debd9-17fb-4a7a-bac7-3c2a5952283a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:46.496702 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1debd9-17fb-4a7a-bac7-3c2a5952283a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:46.509376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-dd1debd9-17fb-4a7a-bac7-3c2a5952283a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:46.509376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:46.509376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:46.509376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:46.509376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:46.509376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:46.509376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:46.509376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:46.509376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:46.509376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:46.509376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:46.509376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:46.509376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:46.509376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:46.509376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:46.510369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:46.510369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:46.510369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:46.510369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:46.510369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:46.510369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:46.510369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:46.510369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:46.510369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:46.510369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:46.510369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:46.510369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:46.510369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:46.510369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:46.510369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:46.510369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:46.510369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:46.511237 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:46.511237 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:46.511237 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:46.511237 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:46.511237 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:46.511237 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:46.511237 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:46.511237 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:46.511237 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 473/950] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:46.511705 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1debd9-17fb-4a7a-bac7-3c2a5952283a None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:46.542553 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:46.598648 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234e675-81d8-47ce-898a-012a63bfdb64 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:46.616576 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1debd9-17fb-4a7a-bac7-3c2a5952283a None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:46.632321 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea5defe-2df8-48c0-a961-786ddaeaeff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:46.633075 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5defe-2df8-48c0-a961-786ddaeaeff3 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:46.633354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5defe-2df8-48c0-a961-786ddaeaeff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:46.633624 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5defe-2df8-48c0-a961-786ddaeaeff3 None admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:46.634364 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea5defe-2df8-48c0-a961-786ddaeaeff3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:46.634665 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea5defe-2df8-48c0-a961-786ddaeaeff3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:46.636690 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea5defe-2df8-48c0-a961-786ddaeaeff3 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:46.651029 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 478/951] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:46.651736 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea5defe-2df8-48c0-a961-786ddaeaeff3 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:46.657165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43323aa-f995-4b59-b35c-ee22571661d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:46.657793 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43323aa-f995-4b59-b35c-ee22571661d5 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:46.658070 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43323aa-f995-4b59-b35c-ee22571661d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:46.658339 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43323aa-f995-4b59-b35c-ee22571661d5 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:46.658990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43323aa-f995-4b59-b35c-ee22571661d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:46.660353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43323aa-f995-4b59-b35c-ee22571661d5 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:46.676954 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 474/952] 10.222.0.74 () {56 vars in 1087 bytes} [Wed Jul 27 16:23:46 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:23:46.676954 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43323aa-f995-4b59-b35c-ee22571661d5 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:46.716513 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:46.783906 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea5defe-2df8-48c0-a961-786ddaeaeff3 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:46.786538 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:46.798090 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43323aa-f995-4b59-b35c-ee22571661d5 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:46.823096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7491da2-8ba8-4b3c-9fc1-02c13ff28c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:46.823820 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7491da2-8ba8-4b3c-9fc1-02c13ff28c3f None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:46.824172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7491da2-8ba8-4b3c-9fc1-02c13ff28c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:46.824517 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7491da2-8ba8-4b3c-9fc1-02c13ff28c3f None admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:46.825352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7491da2-8ba8-4b3c-9fc1-02c13ff28c3f None admin] RBAC: Adding query filter params (name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:46.825975 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7491da2-8ba8-4b3c-9fc1-02c13ff28c3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:46.827500 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7491da2-8ba8-4b3c-9fc1-02c13ff28c3f None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:46.843921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c5186-f52d-45f3-9a4d-4746df24e127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:46.844914 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c5186-f52d-45f3-9a4d-4746df24e127 None admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:46.845210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c5186-f52d-45f3-9a4d-4746df24e127 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:46.845487 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c5186-f52d-45f3-9a4d-4746df24e127 None admin] PATH_INFO: `/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/07d6bffc69674103b4c6540598a780b6/roles/2bafbadedabe4d7681043ae2e7cadacd` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:46.847920 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c5186-f52d-45f3-9a4d-4746df24e127 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d6bcdd09cf24a5c8003951db37020cf, user_id=07d6bffc69674103b4c6540598a780b6, role_id=2bafbadedabe4d7681043ae2e7cadacd)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:46.849496 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c5186-f52d-45f3-9a4d-4746df24e127 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:46.853179 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 479/953] 10.222.0.74 () {56 vars in 1112 bytes} [Wed Jul 27 16:23:46 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:46.866691 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 475/954] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:23:46 2022] PUT /identity/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/07d6bffc69674103b4c6540598a780b6/roles/2bafbadedabe4d7681043ae2e7cadacd => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:23:47.080175 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07436e55-9b60-454d-b363-fc599082297d None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:47.080564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07436e55-9b60-454d-b363-fc599082297d None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:47.080747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07436e55-9b60-454d-b363-fc599082297d None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:47.082427 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 480/955] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:47.087637 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9edb4-6bf1-4342-9c94-c7abba2d006d None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:47.087926 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9edb4-6bf1-4342-9c94-c7abba2d006d None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:47.088303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9edb4-6bf1-4342-9c94-c7abba2d006d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:47.109192 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-a7b9edb4-6bf1-4342-9c94-c7abba2d006d None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:47.110628 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-a7b9edb4-6bf1-4342-9c94-c7abba2d006d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:47.180993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 476/956] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:47.204273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07436e55-9b60-454d-b363-fc599082297d None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:47.314832 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07436e55-9b60-454d-b363-fc599082297d None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:47.351225 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f93642-8f5e-404f-b249-18cbf3728343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:47.351841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f93642-8f5e-404f-b249-18cbf3728343 None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:47.352118 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f93642-8f5e-404f-b249-18cbf3728343 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:47.352389 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f93642-8f5e-404f-b249-18cbf3728343 None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:47.353111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f93642-8f5e-404f-b249-18cbf3728343 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:47.355116 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f93642-8f5e-404f-b249-18cbf3728343 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:47.371502 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 481/957] 10.222.0.74 () {60 vars in 1141 bytes} [Wed Jul 27 16:23:47 2022] POST /identity/v3/services => generated 242 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:23:48.787906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67167a-0450-48cf-80ff-2f613f843047 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:48.789102 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67167a-0450-48cf-80ff-2f613f843047 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:48.789616 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67167a-0450-48cf-80ff-2f613f843047 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:48.791480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 477/958] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:48.796797 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e017c-d9c6-4106-b526-a52f46139185 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:48.797648 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e017c-d9c6-4106-b526-a52f46139185 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:48.798145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e017c-d9c6-4106-b526-a52f46139185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:48.817028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-6b1e017c-d9c6-4106-b526-a52f46139185 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:48.818500 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-6b1e017c-d9c6-4106-b526-a52f46139185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:48.837744 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:48.900152 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 482/959] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:48.920346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb88350-ac9f-4d69-8ab1-4a07d3fb5f69 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:48.920656 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb88350-ac9f-4d69-8ab1-4a07d3fb5f69 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:48.920937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb88350-ac9f-4d69-8ab1-4a07d3fb5f69 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:48.922776 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 478/960] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:48.924519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e017c-d9c6-4106-b526-a52f46139185 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:48.928095 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4cc12-8a4c-4a1a-bdfc-4065d69a660a None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:48.928647 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4cc12-8a4c-4a1a-bdfc-4065d69a660a None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:48.929213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4cc12-8a4c-4a1a-bdfc-4065d69a660a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:48.949002 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-fca4cc12-8a4c-4a1a-bdfc-4065d69a660a None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:48.950825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-fca4cc12-8a4c-4a1a-bdfc-4065d69a660a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:48.982878 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:49.025260 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 479/961] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:49.027344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ade9ca-0069-4ebf-aeaa-18e3e42be847 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:49.027947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ade9ca-0069-4ebf-aeaa-18e3e42be847 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:49.028708 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ade9ca-0069-4ebf-aeaa-18e3e42be847 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:49.031511 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 480/962] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:49.039790 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e6b80-0269-410c-adb4-9c6bdd9d8e5d None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:49.040001 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e6b80-0269-410c-adb4-9c6bdd9d8e5d None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:49.040447 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e6b80-0269-410c-adb4-9c6bdd9d8e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:49.045793 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e017c-d9c6-4106-b526-a52f46139185 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:49.060380 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-a45e6b80-0269-410c-adb4-9c6bdd9d8e5d None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:49.061591 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-a45e6b80-0269-410c-adb4-9c6bdd9d8e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:49.084321 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ab1d2-a07e-4014-89b0-a9b9f213a702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:49.085090 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ab1d2-a07e-4014-89b0-a9b9f213a702 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:49.085369 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ab1d2-a07e-4014-89b0-a9b9f213a702 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:49.085649 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ab1d2-a07e-4014-89b0-a9b9f213a702 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:49.086267 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ab1d2-a07e-4014-89b0-a9b9f213a702 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:49.087637 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ab1d2-a07e-4014-89b0-a9b9f213a702 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:49.097165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-467ab1d2-a07e-4014-89b0-a9b9f213a702 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 16:23:49.097165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:49.097165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:23:49.097165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:23:49.097165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:23:49.097165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:23:49.097165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:23:49.097165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:49.097165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 16:23:49.097165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:49.097165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:49.097165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:49.097165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:49.097165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:49.097165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:49.097165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:49.097165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:49.098329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:49.098329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:49.098329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:49.098329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:49.098329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:49.098329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:49.098329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:23:49.098329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:23:49.098329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:23:49.098329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:23:49.098329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:49.098329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:49.098329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:49.098329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:49.098329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:49.098329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:49.099215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:49.099215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:49.099215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:49.099215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:49.099215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:49.099215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:49.099215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:49.099215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:49.099215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:49.099215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:49.099215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:23:49.099215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:49.099215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 16:23:49.099215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:49.099215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 483/963] 10.222.0.74 () {56 vars in 1126 bytes} [Wed Jul 27 16:23:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:49.100078 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ab1d2-a07e-4014-89b0-a9b9f213a702 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:49.116228 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:49.141101 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 481/964] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:49.141101 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e6b80-0269-410c-adb4-9c6bdd9d8e5d None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:49.220730 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ab1d2-a07e-4014-89b0-a9b9f213a702 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:49.237867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:49.253520 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e6b80-0269-410c-adb4-9c6bdd9d8e5d None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:49.264758 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f371d7a-e90c-4398-b432-632b081de2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:49.265922 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f371d7a-e90c-4398-b432-632b081de2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:49.266382 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f371d7a-e90c-4398-b432-632b081de2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:49.266966 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f371d7a-e90c-4398-b432-632b081de2c8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:49.280426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f371d7a-e90c-4398-b432-632b081de2c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:49.288607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-4f371d7a-e90c-4398-b432-632b081de2c8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 16:23:49.288607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:49.288607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:49.288607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:49.288607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:49.288607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:49.288607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:49.288607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:49.288607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:49.288607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:49.288607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:49.288607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:49.288607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:49.288607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:49.288607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:49.288607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:49.289860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:49.289860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:49.289860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:49.289860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:49.289860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:49.289860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:49.289860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:49.289860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:49.289860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:49.289860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:49.289860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:49.289860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:49.289860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:49.289860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:49.289860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:49.289860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:49.290902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:49.290902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:49.290902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:49.290902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:49.290902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:49.290902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:49.290902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:49.290902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:49.290902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:49.290902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:49.290902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 16:23:49.290902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:49.292490 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49f409-6aaf-4a73-982c-97bfe0b0868d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:49.293128 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49f409-6aaf-4a73-982c-97bfe0b0868d None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:49.293405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49f409-6aaf-4a73-982c-97bfe0b0868d None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:49.293674 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49f409-6aaf-4a73-982c-97bfe0b0868d None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:49.294351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49f409-6aaf-4a73-982c-97bfe0b0868d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:49.294716 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 484/965] 10.222.0.74 () {56 vars in 1120 bytes} [Wed Jul 27 16:23:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:49.294830 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49f409-6aaf-4a73-982c-97bfe0b0868d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:49.295618 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f371d7a-e90c-4398-b432-632b081de2c8 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:49.296301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49f409-6aaf-4a73-982c-97bfe0b0868d None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:49.305964 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 482/966] 10.222.0.74 () {56 vars in 1122 bytes} [Wed Jul 27 16:23:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:49.305964 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49f409-6aaf-4a73-982c-97bfe0b0868d None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:49.376388 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:49.404277 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f371d7a-e90c-4398-b432-632b081de2c8 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:49.415485 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49f409-6aaf-4a73-982c-97bfe0b0868d None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:49.439244 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d3166-cd68-44dc-9c4a-e481dfc7cb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:49.439858 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d3166-cd68-44dc-9c4a-e481dfc7cb94 None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:49.440135 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d3166-cd68-44dc-9c4a-e481dfc7cb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:49.440405 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d3166-cd68-44dc-9c4a-e481dfc7cb94 None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:49.441105 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d3166-cd68-44dc-9c4a-e481dfc7cb94 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:49.442525 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d3166-cd68-44dc-9c4a-e481dfc7cb94 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:49.445061 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: WARNING keystone.server.flask.common [None req-612d3166-cd68-44dc-9c4a-e481dfc7cb94 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 27 16:23:49.452643 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761806e-41c6-42ac-965d-e7eb9e41d17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:49.453705 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761806e-41c6-42ac-965d-e7eb9e41d17e None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:49.453988 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761806e-41c6-42ac-965d-e7eb9e41d17e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:49.454260 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761806e-41c6-42ac-965d-e7eb9e41d17e None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:49.454984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761806e-41c6-42ac-965d-e7eb9e41d17e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:49.455293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761806e-41c6-42ac-965d-e7eb9e41d17e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:49.456784 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761806e-41c6-42ac-965d-e7eb9e41d17e None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:49.457830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.sql.core [None req-612d3166-cd68-44dc-9c4a-e481dfc7cb94 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 16:23:49.457830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 27 16:23:49.457830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [parameters: {'user_id': 'f5e13d3ec1ce4ac1a5710dbbeb123597', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 27 16:23:49.457830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80109) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 16:23:49.465232 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 483/967] 10.222.0.74 () {56 vars in 1115 bytes} [Wed Jul 27 16:23:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:49.465937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761806e-41c6-42ac-965d-e7eb9e41d17e None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:49.466207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-612d3166-cd68-44dc-9c4a-e481dfc7cb94 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 27 16:23:49.466207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:49.466207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 16:23:49.466207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 16:23:49.466207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 16:23:49.466207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 16:23:49.466207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 16:23:49.466207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 16:23:49.466207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 16:23:49.466207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application conn.query(q) Jul 27 16:23:49.466207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 16:23:49.466207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 16:23:49.466207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 16:23:49.466207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application result.read() Jul 27 16:23:49.466207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 16:23:49.467051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 16:23:49.467051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 16:23:49.467051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 16:23:49.467051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 16:23:49.467051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 16:23:49.467051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 16:23:49.467051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 16:23:49.467051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 16:23:49.467051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:49.467051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 16:23:49.467051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:49.467051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:49.467051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 16:23:49.467051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 16:23:49.467051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 27 16:23:49.467051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 27 16:23:49.467051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 16:23:49.467051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application next(self.gen) Jul 27 16:23:49.473173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 16:23:49.473173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application yield resource Jul 27 16:23:49.473173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 16:23:49.473173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application next(self.gen) Jul 27 16:23:49.473173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 16:23:49.473173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 16:23:49.473173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 16:23:49.473173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 16:23:49.473173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 16:23:49.473173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise self.value Jul 27 16:23:49.473173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 16:23:49.473173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 16:23:49.473173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 16:23:49.473173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application session.commit() Jul 27 16:23:49.473173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 16:23:49.473173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 16:23:49.473173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 16:23:49.474239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 16:23:49.474239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 16:23:49.474239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self.session.flush() Jul 27 16:23:49.474239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 16:23:49.474239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 16:23:49.474239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 16:23:49.474239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 16:23:49.474239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 16:23:49.474239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application compat.raise_( Jul 27 16:23:49.474239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 16:23:49.474239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception Jul 27 16:23:49.474239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 16:23:49.474239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 16:23:49.474239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 16:23:49.474239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 16:23:49.474239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 16:23:49.474239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 16:23:49.475165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 16:23:49.475165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 16:23:49.475165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 27 16:23:49.475165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 27 16:23:49.475165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 16:23:49.475165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 16:23:49.475165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 16:23:49.475165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 16:23:49.475165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 16:23:49.475165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 16:23:49.475165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 16:23:49.475165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 16:23:49.475165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 16:23:49.475165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 16:23:49.475165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 16:23:49.475165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception Jul 27 16:23:49.476709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 16:23:49.476709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 16:23:49.476709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 16:23:49.476709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 16:23:49.476709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 16:23:49.476709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 16:23:49.476709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 16:23:49.476709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application conn.query(q) Jul 27 16:23:49.476709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 16:23:49.476709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 16:23:49.476709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 16:23:49.476709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application result.read() Jul 27 16:23:49.476709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 16:23:49.476709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 16:23:49.476709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 16:23:49.476709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 16:23:49.478210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 16:23:49.478210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 16:23:49.478210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 16:23:49.478210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 16:23:49.478210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 16:23:49.478210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 27 16:23:49.478210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f5e13d3ec1ce4ac1a5710dbbeb123597', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 27 16:23:49.478210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 16:23:49.478210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:49.478210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:49.478210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:49.478210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:49.478210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:49.478210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:49.478210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:49.480234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:49.480234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:49.480234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:49.480234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:49.480234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:49.480234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:49.480234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:49.480234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 27 16:23:49.480234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 27 16:23:49.480234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:49.480234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:49.480234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:49.480234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:49.480234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:49.480234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:49.480234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 27 16:23:49.480234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 27 16:23:49.481514 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 27 16:23:49.481514 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 27 16:23:49.481514 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 16:23:49.481514 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 16:23:49.481514 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 27 16:23:49.481514 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:49.481514 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 485/968] 10.222.0.74 () {60 vars in 1131 bytes} [Wed Jul 27 16:23:49 2022] POST /identity/v3/users => generated 160 bytes in 172 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:49.481514 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45146af9-904f-4888-913e-cfb9265a0f4c None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:49.481514 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45146af9-904f-4888-913e-cfb9265a0f4c None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:49.481514 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45146af9-904f-4888-913e-cfb9265a0f4c None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:49.482343 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 486/969] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:49.482343 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45146af9-904f-4888-913e-cfb9265a0f4c None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:49.485149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:49.568688 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761806e-41c6-42ac-965d-e7eb9e41d17e None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:49.577656 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45146af9-904f-4888-913e-cfb9265a0f4c None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:49.600994 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62809855-ff1a-419c-b2a6-e573be2fe368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:49.601603 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62809855-ff1a-419c-b2a6-e573be2fe368 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:49.601897 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62809855-ff1a-419c-b2a6-e573be2fe368 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:49.602176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62809855-ff1a-419c-b2a6-e573be2fe368 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:49.602836 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62809855-ff1a-419c-b2a6-e573be2fe368 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:49.603142 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62809855-ff1a-419c-b2a6-e573be2fe368 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:49.604550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62809855-ff1a-419c-b2a6-e573be2fe368 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:49.609211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 484/970] 10.222.0.74 () {56 vars in 1122 bytes} [Wed Jul 27 16:23:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:49.609977 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62809855-ff1a-419c-b2a6-e573be2fe368 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:49.610923 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae28b3-5fac-4931-bc36-22f5292bfb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:49.611563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae28b3-5fac-4931-bc36-22f5292bfb22 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:49.611842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae28b3-5fac-4931-bc36-22f5292bfb22 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:49.612114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae28b3-5fac-4931-bc36-22f5292bfb22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:49.615070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:49.624838 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae28b3-5fac-4931-bc36-22f5292bfb22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:49.626439 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae28b3-5fac-4931-bc36-22f5292bfb22 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:49.640798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-63ae28b3-5fac-4931-bc36-22f5292bfb22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:49.640798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:49.640798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:49.640798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:49.640798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:49.640798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:49.640798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:49.640798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:49.640798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:49.640798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:49.640798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:49.640798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:49.640798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:49.640798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:49.640798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:49.641703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:49.641703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:49.641703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:49.641703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:49.641703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:49.641703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:49.641703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:49.641703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:49.641703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:49.641703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:49.641703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:49.641703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:49.641703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:49.641703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:49.641703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:49.641703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:49.641703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:49.642612 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:49.642612 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:49.642612 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:49.642612 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:49.642612 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 487/971] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:49.643114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b415267-b2ad-458c-b419-fdc852c2cda7 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:49.643384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b415267-b2ad-458c-b419-fdc852c2cda7 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:49.643611 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b415267-b2ad-458c-b419-fdc852c2cda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:49.661049 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-2b415267-b2ad-458c-b419-fdc852c2cda7 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:49.662635 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-2b415267-b2ad-458c-b419-fdc852c2cda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:49.692771 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:49.716107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62809855-ff1a-419c-b2a6-e573be2fe368 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:49.729163 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 488/972] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:49.729957 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b415267-b2ad-458c-b419-fdc852c2cda7 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:49.749846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245067ed-22d0-49be-9053-29c447a07e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:49.750492 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245067ed-22d0-49be-9053-29c447a07e5b None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:49.750774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245067ed-22d0-49be-9053-29c447a07e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:49.751048 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245067ed-22d0-49be-9053-29c447a07e5b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:49.758576 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245067ed-22d0-49be-9053-29c447a07e5b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:49.760080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245067ed-22d0-49be-9053-29c447a07e5b None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:49.771301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-245067ed-22d0-49be-9053-29c447a07e5b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 16:23:49.771301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:49.771301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:49.771301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:49.771301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:49.771301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:49.771301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:49.771301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:49.771301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:49.771301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:49.771301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:49.771301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:49.771301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:49.771301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:49.771301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:49.771301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:49.772428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:49.772428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:49.772428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:49.772428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:49.772428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:49.772428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:49.772428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:49.772428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:49.772428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:49.772428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:49.772428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:49.772428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:49.772428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:49.772428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:49.772428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:49.772428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:49.772428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:49.772428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:49.773420 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:49.773420 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:49.773420 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:49.773420 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:49.773420 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:49.773420 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:49.773420 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:49.773420 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:49.773420 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 16:23:49.773420 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:49.773420 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 485/973] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:23:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:49.773420 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245067ed-22d0-49be-9053-29c447a07e5b None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:49.830758 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b415267-b2ad-458c-b419-fdc852c2cda7 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:49.847893 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:49.865479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e550e-259d-43b2-aced-483808b9eead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:49.866099 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e550e-259d-43b2-aced-483808b9eead None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:49.866379 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e550e-259d-43b2-aced-483808b9eead None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:49.866653 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e550e-259d-43b2-aced-483808b9eead None admin] PATH_INFO: `/v3/endpoints` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:49.867345 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e550e-259d-43b2-aced-483808b9eead None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:49.868714 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e550e-259d-43b2-aced-483808b9eead None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:49.870954 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245067ed-22d0-49be-9053-29c447a07e5b None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:49.891284 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 489/974] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:23:49 2022] POST /identity/v3/endpoints => generated 349 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:23:49.891937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e550e-259d-43b2-aced-483808b9eead None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:49.919384 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:49.926021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2232fd4-0c1a-4146-80f3-022c386384b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:49.926849 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2232fd4-0c1a-4146-80f3-022c386384b2 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:49.927119 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2232fd4-0c1a-4146-80f3-022c386384b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:49.927445 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2232fd4-0c1a-4146-80f3-022c386384b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:49.928332 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2232fd4-0c1a-4146-80f3-022c386384b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:49.928980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2232fd4-0c1a-4146-80f3-022c386384b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:49.931023 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2232fd4-0c1a-4146-80f3-022c386384b2 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:49.951338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 486/975] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:49.951338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2232fd4-0c1a-4146-80f3-022c386384b2 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:50.019922 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e550e-259d-43b2-aced-483808b9eead None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:50.064785 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96110c90-e40b-4461-9f51-e4dbb3fb113a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:50.065464 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96110c90-e40b-4461-9f51-e4dbb3fb113a None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:50.065738 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96110c90-e40b-4461-9f51-e4dbb3fb113a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:50.066049 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96110c90-e40b-4461-9f51-e4dbb3fb113a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:50.066621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96110c90-e40b-4461-9f51-e4dbb3fb113a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:50.068471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96110c90-e40b-4461-9f51-e4dbb3fb113a None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:50.068599 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2232fd4-0c1a-4146-80f3-022c386384b2 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:50.079166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-96110c90-e40b-4461-9f51-e4dbb3fb113a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 16:23:50.079166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:50.079166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:23:50.079166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:23:50.079166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:23:50.079166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:23:50.079166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:23:50.079166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:50.079166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 16:23:50.079166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:50.079166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:50.079166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:50.079166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:50.079166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:50.079166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:50.079166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:50.079166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:50.080170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:50.080170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:50.080170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:50.080170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:50.080170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:50.080170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:50.080170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:23:50.080170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:23:50.080170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:23:50.080170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:23:50.080170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:50.080170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:50.080170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:50.080170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:50.080170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:50.080170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:50.081058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:50.081058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:50.081058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:50.081058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:50.081058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:50.081058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:50.081058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:50.081058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:50.081058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:50.081058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:50.081058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:23:50.081058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:50.081058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 16:23:50.081058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:50.081058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 490/976] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:50.082406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96110c90-e40b-4461-9f51-e4dbb3fb113a None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:50.111316 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035137a8-1d4f-4658-bcbc-7314767d40f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:50.111945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035137a8-1d4f-4658-bcbc-7314767d40f8 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:50.112230 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035137a8-1d4f-4658-bcbc-7314767d40f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:50.112506 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035137a8-1d4f-4658-bcbc-7314767d40f8 None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:50.113304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035137a8-1d4f-4658-bcbc-7314767d40f8 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:50.113613 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035137a8-1d4f-4658-bcbc-7314767d40f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:50.115045 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035137a8-1d4f-4658-bcbc-7314767d40f8 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:50.126250 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:50.154765 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 487/977] 10.222.0.74 () {56 vars in 1101 bytes} [Wed Jul 27 16:23:49 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:50.155664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035137a8-1d4f-4658-bcbc-7314767d40f8 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:50.211454 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96110c90-e40b-4461-9f51-e4dbb3fb113a None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:50.237630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:50.254265 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f4d25-1475-451a-8e8e-76a6311ce3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:50.255378 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f4d25-1475-451a-8e8e-76a6311ce3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:50.256048 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f4d25-1475-451a-8e8e-76a6311ce3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:50.256527 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f4d25-1475-451a-8e8e-76a6311ce3e6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:50.265865 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f4d25-1475-451a-8e8e-76a6311ce3e6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:50.267728 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f4d25-1475-451a-8e8e-76a6311ce3e6 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:50.281584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-432f4d25-1475-451a-8e8e-76a6311ce3e6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 16:23:50.281584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:50.281584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:50.281584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:50.281584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:50.281584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:50.281584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:50.281584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:50.281584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:50.281584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:50.281584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:50.281584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:50.281584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:50.281584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:50.281584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:50.281584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:50.282658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:50.282658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:50.282658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:50.282658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:50.282658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:50.282658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:50.282658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:50.282658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:50.282658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:50.282658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:50.282658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:50.282658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:50.282658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:50.282658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:50.282658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:50.282658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:50.282658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:50.282658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:50.283582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:50.283582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:50.283582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:50.283582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:50.283582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:50.283582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:50.283582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:50.283582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:50.283582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 16:23:50.283582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:50.290976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 491/978] 10.222.0.74 () {56 vars in 1117 bytes} [Wed Jul 27 16:23:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:50.292333 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f4d25-1475-451a-8e8e-76a6311ce3e6 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:50.309023 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035137a8-1d4f-4658-bcbc-7314767d40f8 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:50.363229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75404ec3-75c8-4534-932f-e4ee0fc8df73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:50.363783 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75404ec3-75c8-4534-932f-e4ee0fc8df73 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:50.364067 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75404ec3-75c8-4534-932f-e4ee0fc8df73 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:50.364343 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75404ec3-75c8-4534-932f-e4ee0fc8df73 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:50.365076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75404ec3-75c8-4534-932f-e4ee0fc8df73 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:50.365370 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75404ec3-75c8-4534-932f-e4ee0fc8df73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:50.366775 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75404ec3-75c8-4534-932f-e4ee0fc8df73 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:50.372351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 488/979] 10.222.0.74 () {56 vars in 1112 bytes} [Wed Jul 27 16:23:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:50.379088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75404ec3-75c8-4534-932f-e4ee0fc8df73 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:50.420222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:50.426508 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f4d25-1475-451a-8e8e-76a6311ce3e6 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:50.470804 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bcf28a-bb8b-45bc-bc09-53eb66707d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:50.471450 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcf28a-bb8b-45bc-bc09-53eb66707d1d None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:50.471736 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcf28a-bb8b-45bc-bc09-53eb66707d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:50.472011 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcf28a-bb8b-45bc-bc09-53eb66707d1d None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:50.472810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bcf28a-bb8b-45bc-bc09-53eb66707d1d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:50.473149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bcf28a-bb8b-45bc-bc09-53eb66707d1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:50.474570 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bcf28a-bb8b-45bc-bc09-53eb66707d1d None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:50.493296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 492/980] 10.222.0.74 () {56 vars in 1149 bytes} [Wed Jul 27 16:23:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:50.499027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bcf28a-bb8b-45bc-bc09-53eb66707d1d None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:50.502187 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75404ec3-75c8-4534-932f-e4ee0fc8df73 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:50.517910 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:50.542007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0b4c0-0a7c-418e-9fb9-01444fb24edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:50.542633 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0b4c0-0a7c-418e-9fb9-01444fb24edb None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:50.542917 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0b4c0-0a7c-418e-9fb9-01444fb24edb None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:50.543196 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0b4c0-0a7c-418e-9fb9-01444fb24edb None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:50.544230 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0b4c0-0a7c-418e-9fb9-01444fb24edb None admin] RBAC: Adding query filter params (type=compute) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:50.544535 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0b4c0-0a7c-418e-9fb9-01444fb24edb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:50.546242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0b4c0-0a7c-418e-9fb9-01444fb24edb None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:50.552052 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 489/981] 10.222.0.74 () {56 vars in 1112 bytes} [Wed Jul 27 16:23:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:50.557734 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0b4c0-0a7c-418e-9fb9-01444fb24edb None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:50.606189 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bcf28a-bb8b-45bc-bc09-53eb66707d1d None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:50.643635 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae81e1-064e-4906-a426-b93ea330201b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:50.644155 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae81e1-064e-4906-a426-b93ea330201b None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:50.644432 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae81e1-064e-4906-a426-b93ea330201b None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:50.644703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae81e1-064e-4906-a426-b93ea330201b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:50.664663 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0b4c0-0a7c-418e-9fb9-01444fb24edb None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:50.671803 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:50.701492 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dc1ca-9595-40ba-8b6f-c4edf1d0092f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:50.702111 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dc1ca-9595-40ba-8b6f-c4edf1d0092f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:50.702392 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dc1ca-9595-40ba-8b6f-c4edf1d0092f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:50.702665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dc1ca-9595-40ba-8b6f-c4edf1d0092f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:50.703416 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550dc1ca-9595-40ba-8b6f-c4edf1d0092f None admin] RBAC: Adding query filter params (interface=public, service_id=5b0d719a3c984d2c9c4f2df930fc8c17, region_id=RegionOne) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:50.703723 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550dc1ca-9595-40ba-8b6f-c4edf1d0092f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:50.705107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550dc1ca-9595-40ba-8b6f-c4edf1d0092f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:50.710032 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 490/982] 10.222.0.74 () {56 vars in 1251 bytes} [Wed Jul 27 16:23:50 2022] GET /identity/v3/endpoints?service_id=5b0d719a3c984d2c9c4f2df930fc8c17&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:50.715804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dc1ca-9595-40ba-8b6f-c4edf1d0092f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:50.726603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae81e1-064e-4906-a426-b93ea330201b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:50.728239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae81e1-064e-4906-a426-b93ea330201b None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:50.759707 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:50.814729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dc1ca-9595-40ba-8b6f-c4edf1d0092f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:50.835268 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 493/983] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:23:50 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 16:23:50.840508 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae81e1-064e-4906-a426-b93ea330201b None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:50.852981 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecee630-fa8a-43dd-8e2f-bbd1f48d341f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:50.853603 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecee630-fa8a-43dd-8e2f-bbd1f48d341f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:50.854449 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecee630-fa8a-43dd-8e2f-bbd1f48d341f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:50.854788 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecee630-fa8a-43dd-8e2f-bbd1f48d341f None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:50.855379 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecee630-fa8a-43dd-8e2f-bbd1f48d341f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:50.856971 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecee630-fa8a-43dd-8e2f-bbd1f48d341f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:50.871943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 491/984] 10.222.0.74 () {56 vars in 1087 bytes} [Wed Jul 27 16:23:50 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:23:50.911859 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:50.961078 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae81e1-064e-4906-a426-b93ea330201b None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:50.998446 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b274d-9739-45f0-a3fe-8557b82e7ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:50.999107 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b274d-9739-45f0-a3fe-8557b82e7ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:50.999386 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b274d-9739-45f0-a3fe-8557b82e7ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:50.999666 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b274d-9739-45f0-a3fe-8557b82e7ad5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:51.010936 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b274d-9739-45f0-a3fe-8557b82e7ad5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:51.012401 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b274d-9739-45f0-a3fe-8557b82e7ad5 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:51.030856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-209b274d-9739-45f0-a3fe-8557b82e7ad5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:51.030856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:51.030856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:51.030856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:51.030856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:51.030856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:51.030856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:51.030856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:51.030856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:51.030856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:51.030856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:51.030856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:51.030856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:51.030856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:51.030856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:51.031842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:51.031842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:51.031842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:51.031842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:51.031842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:51.031842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:51.031842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:51.031842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:51.031842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:51.031842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:51.031842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:51.031842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:51.031842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:51.031842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:51.031842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:51.031842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:51.031842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:51.032840 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:51.032840 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:51.032840 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:51.032840 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:51.032840 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 494/985] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:23:51.038285 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecee630-fa8a-43dd-8e2f-bbd1f48d341f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:51.144640 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecee630-fa8a-43dd-8e2f-bbd1f48d341f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:51.154496 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:51.185747 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d5f69-7c9f-4e90-9214-515b45b2ab48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:51.186368 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d5f69-7c9f-4e90-9214-515b45b2ab48 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:51.186646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d5f69-7c9f-4e90-9214-515b45b2ab48 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:51.186915 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d5f69-7c9f-4e90-9214-515b45b2ab48 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:51.187596 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d5f69-7c9f-4e90-9214-515b45b2ab48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:51.187896 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d5f69-7c9f-4e90-9214-515b45b2ab48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:51.189282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d5f69-7c9f-4e90-9214-515b45b2ab48 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:51.201678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 492/986] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:51.206921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b274d-9739-45f0-a3fe-8557b82e7ad5 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:51.309626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b274d-9739-45f0-a3fe-8557b82e7ad5 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:51.318981 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:51.347101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889f246-2478-4f5b-9f1a-dc5fdd624cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:51.347738 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889f246-2478-4f5b-9f1a-dc5fdd624cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:51.348014 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889f246-2478-4f5b-9f1a-dc5fdd624cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:51.348293 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889f246-2478-4f5b-9f1a-dc5fdd624cc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:51.354925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889f246-2478-4f5b-9f1a-dc5fdd624cc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:51.356356 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889f246-2478-4f5b-9f1a-dc5fdd624cc6 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:51.367576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-2889f246-2478-4f5b-9f1a-dc5fdd624cc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:51.367576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:51.367576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:51.367576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:51.367576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:51.367576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:51.367576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:51.367576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:51.367576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:51.367576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:51.367576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:51.367576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:51.367576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:51.367576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:51.367576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:51.368470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:51.368470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:51.368470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:51.368470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:51.368470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:51.368470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:51.368470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:51.368470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:51.368470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:51.368470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:51.368470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:51.368470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:51.368470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:51.368470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:51.368470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:51.368470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:51.368470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:51.369926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:51.369926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:51.369926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:51.369926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:51.369926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:51.369926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:51.369926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:51.369926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:51.373216 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 495/987] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:51.374891 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d5f69-7c9f-4e90-9214-515b45b2ab48 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:51.476397 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d5f69-7c9f-4e90-9214-515b45b2ab48 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:51.511912 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac6de5-06cf-4dfa-953c-8acea593db85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:51.512533 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac6de5-06cf-4dfa-953c-8acea593db85 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:51.512818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac6de5-06cf-4dfa-953c-8acea593db85 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:51.513160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac6de5-06cf-4dfa-953c-8acea593db85 None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:51.513849 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac6de5-06cf-4dfa-953c-8acea593db85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:51.514153 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac6de5-06cf-4dfa-953c-8acea593db85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:51.515584 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac6de5-06cf-4dfa-953c-8acea593db85 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:51.528705 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 493/988] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:51.534416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889f246-2478-4f5b-9f1a-dc5fdd624cc6 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:51.641184 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889f246-2478-4f5b-9f1a-dc5fdd624cc6 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:51.676241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ea2a3-5a57-437b-bf26-3d4e5cbb9e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:51.676849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ea2a3-5a57-437b-bf26-3d4e5cbb9e64 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:51.677154 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ea2a3-5a57-437b-bf26-3d4e5cbb9e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:51.677424 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ea2a3-5a57-437b-bf26-3d4e5cbb9e64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:51.678196 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ea2a3-5a57-437b-bf26-3d4e5cbb9e64 None admin] RBAC: Adding query filter params (role.id=2bafbadedabe4d7681043ae2e7cadacd, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=07d6bffc69674103b4c6540598a780b6) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:51.678499 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ea2a3-5a57-437b-bf26-3d4e5cbb9e64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:51.680029 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ea2a3-5a57-437b-bf26-3d4e5cbb9e64 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:51.686664 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 496/989] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:23:51 2022] GET /identity/v3/role_assignments?user.id=07d6bffc69674103b4c6540598a780b6&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=2bafbadedabe4d7681043ae2e7cadacd => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:51.977687 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed39b6-6d5d-4f69-95d7-622d7f9e3754 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:51.978258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed39b6-6d5d-4f69-95d7-622d7f9e3754 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:51.978258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed39b6-6d5d-4f69-95d7-622d7f9e3754 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:51.980258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 494/990] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:51.986234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8b36e-8927-45dc-88ab-1f6f1d137d7c None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:51.986749 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8b36e-8927-45dc-88ab-1f6f1d137d7c None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:51.986827 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8b36e-8927-45dc-88ab-1f6f1d137d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:52.006313 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-f8c8b36e-8927-45dc-88ab-1f6f1d137d7c None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:52.007507 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f8c8b36e-8927-45dc-88ab-1f6f1d137d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:52.036545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:52.082851 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 497/991] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:51 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:52.107368 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed39b6-6d5d-4f69-95d7-622d7f9e3754 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:52.139520 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:52.227795 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed39b6-6d5d-4f69-95d7-622d7f9e3754 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:52.262106 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2efa4-0dc8-4d46-ba1b-2c390ce63dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:52.262768 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2efa4-0dc8-4d46-ba1b-2c390ce63dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:52.263050 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2efa4-0dc8-4d46-ba1b-2c390ce63dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:52.263324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2efa4-0dc8-4d46-ba1b-2c390ce63dc4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:52.263945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2efa4-0dc8-4d46-ba1b-2c390ce63dc4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:52.265358 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2efa4-0dc8-4d46-ba1b-2c390ce63dc4 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:52.274189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-0eb2efa4-0dc8-4d46-ba1b-2c390ce63dc4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 16:23:52.274189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:52.274189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:23:52.274189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:23:52.274189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:23:52.274189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:23:52.274189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:23:52.274189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:52.274189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 16:23:52.274189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:52.274189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:52.274189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:52.274189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:52.274189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:52.274189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:52.274189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:52.274189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:52.275168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:52.275168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:52.275168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:52.275168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:52.275168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:52.275168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:52.275168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:23:52.275168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:23:52.275168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:23:52.275168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:23:52.275168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:52.275168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:52.275168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:52.275168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:52.275168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:52.275168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:52.276625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:52.276625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:52.276625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:52.276625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:52.276625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:52.276625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:52.276625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:52.276625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:52.276625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:52.276625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:52.276625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:23:52.276625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:52.276625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 16:23:52.276625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:52.280869 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 495/992] 10.222.0.74 () {56 vars in 1126 bytes} [Wed Jul 27 16:23:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:52.282221 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8b36e-8927-45dc-88ab-1f6f1d137d7c None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:52.381608 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8b36e-8927-45dc-88ab-1f6f1d137d7c None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:52.415746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6dff57-8d1e-42e6-b588-6214dbb69258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:52.416917 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dff57-8d1e-42e6-b588-6214dbb69258 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:52.417235 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dff57-8d1e-42e6-b588-6214dbb69258 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:52.417507 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dff57-8d1e-42e6-b588-6214dbb69258 None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:52.418179 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6dff57-8d1e-42e6-b588-6214dbb69258 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:52.418484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6dff57-8d1e-42e6-b588-6214dbb69258 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:52.419847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6dff57-8d1e-42e6-b588-6214dbb69258 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:52.425116 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 498/993] 10.222.0.74 () {56 vars in 1122 bytes} [Wed Jul 27 16:23:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:52.430723 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2efa4-0dc8-4d46-ba1b-2c390ce63dc4 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:52.521989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:52.533426 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2efa4-0dc8-4d46-ba1b-2c390ce63dc4 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:52.567577 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025c92b-8ba1-4450-b002-7354a9af56f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:52.568201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025c92b-8ba1-4450-b002-7354a9af56f8 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:52.568483 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025c92b-8ba1-4450-b002-7354a9af56f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:52.568758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025c92b-8ba1-4450-b002-7354a9af56f8 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:52.569460 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9025c92b-8ba1-4450-b002-7354a9af56f8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:52.569766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9025c92b-8ba1-4450-b002-7354a9af56f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:52.571139 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9025c92b-8ba1-4450-b002-7354a9af56f8 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:52.576466 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 496/994] 10.222.0.74 () {56 vars in 1122 bytes} [Wed Jul 27 16:23:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:52.582334 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6dff57-8d1e-42e6-b588-6214dbb69258 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:52.610580 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:52.683708 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6dff57-8d1e-42e6-b588-6214dbb69258 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:52.718161 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef715a1-9ed7-4b04-907c-b72f0326f489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:52.718775 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef715a1-9ed7-4b04-907c-b72f0326f489 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:52.719079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef715a1-9ed7-4b04-907c-b72f0326f489 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:52.719354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef715a1-9ed7-4b04-907c-b72f0326f489 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:52.720129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef715a1-9ed7-4b04-907c-b72f0326f489 None admin] RBAC: Adding query filter params (interface=admin, service_id=9031214db68a40c0b596671be7b70f2c, region_id=RegionOne) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:52.720441 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef715a1-9ed7-4b04-907c-b72f0326f489 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:52.721840 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef715a1-9ed7-4b04-907c-b72f0326f489 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:52.727065 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 499/995] 10.222.0.74 () {56 vars in 1249 bytes} [Wed Jul 27 16:23:52 2022] GET /identity/v3/endpoints?service_id=9031214db68a40c0b596671be7b70f2c&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:52.733147 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025c92b-8ba1-4450-b002-7354a9af56f8 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:52.834010 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025c92b-8ba1-4450-b002-7354a9af56f8 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:52.870110 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd566df6-f788-4780-9a6e-a06d01bd3dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:52.870730 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd566df6-f788-4780-9a6e-a06d01bd3dbc None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:52.871010 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd566df6-f788-4780-9a6e-a06d01bd3dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:52.871283 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd566df6-f788-4780-9a6e-a06d01bd3dbc None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:52.871933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd566df6-f788-4780-9a6e-a06d01bd3dbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:52.873337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd566df6-f788-4780-9a6e-a06d01bd3dbc None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:52.875706 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d8029-a477-469a-8134-1a9abbbd2704 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:52.875992 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d8029-a477-469a-8134-1a9abbbd2704 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:52.876268 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d8029-a477-469a-8134-1a9abbbd2704 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:52.877789 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 500/996] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:23:52.883724 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9a87c-4241-4133-9e1c-af10f232aee3 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:52.884097 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9a87c-4241-4133-9e1c-af10f232aee3 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:52.884267 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9a87c-4241-4133-9e1c-af10f232aee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:52.887981 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 497/997] 10.222.0.74 () {56 vars in 1087 bytes} [Wed Jul 27 16:23:52 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:23:52.905047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-3eb9a87c-4241-4133-9e1c-af10f232aee3 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:52.906278 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-3eb9a87c-4241-4133-9e1c-af10f232aee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:52.981053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 501/998] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:52 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 16:23:53.001104 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd566df6-f788-4780-9a6e-a06d01bd3dbc None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:53.022231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:53.107849 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd566df6-f788-4780-9a6e-a06d01bd3dbc None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:53.141074 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf3df13-cba3-4496-a5e1-277a9c551dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:53.141722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3df13-cba3-4496-a5e1-277a9c551dad None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:53.142005 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3df13-cba3-4496-a5e1-277a9c551dad None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:53.142281 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3df13-cba3-4496-a5e1-277a9c551dad None admin] PATH_INFO: `/v3/services/compute` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:53.142908 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf3df13-cba3-4496-a5e1-277a9c551dad None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:53.144288 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf3df13-cba3-4496-a5e1-277a9c551dad None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:53.155923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-caf3df13-cba3-4496-a5e1-277a9c551dad None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 16:23:53.155923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:53.155923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:23:53.155923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:23:53.155923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:23:53.155923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:23:53.155923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:23:53.155923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:53.155923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 16:23:53.155923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:53.155923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:53.155923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:53.155923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:53.155923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:53.155923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:53.155923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:53.155923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:53.157243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:53.157243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:53.157243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:53.157243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:53.157243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:53.157243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:53.157243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:23:53.157243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:23:53.157243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:23:53.157243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:23:53.157243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:53.157243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:53.157243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:53.157243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:53.157243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:53.157243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:53.158127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:53.158127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:53.158127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:53.158127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:53.158127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:53.158127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:53.158127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:53.158127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:53.158127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:53.158127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:53.158127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:23:53.158127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:53.158127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 16:23:53.158127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:53.158127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 498/999] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:53.167334 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb9a87c-4241-4133-9e1c-af10f232aee3 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:53.189407 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:53.269897 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb9a87c-4241-4133-9e1c-af10f232aee3 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:53.304858 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73da89f-3f19-45d7-b4e9-d504883eed28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:53.305511 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73da89f-3f19-45d7-b4e9-d504883eed28 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:53.305794 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73da89f-3f19-45d7-b4e9-d504883eed28 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:53.306068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73da89f-3f19-45d7-b4e9-d504883eed28 None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:53.306781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73da89f-3f19-45d7-b4e9-d504883eed28 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:53.307074 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73da89f-3f19-45d7-b4e9-d504883eed28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:53.308440 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73da89f-3f19-45d7-b4e9-d504883eed28 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:53.313429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 502/1000] 10.222.0.74 () {56 vars in 1112 bytes} [Wed Jul 27 16:23:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:53.318932 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf3df13-cba3-4496-a5e1-277a9c551dad None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:53.403905 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:53.423065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf3df13-cba3-4496-a5e1-277a9c551dad None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:53.457290 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a08a40-1ccd-45bd-bfd1-a7f275bb6b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:53.457902 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a08a40-1ccd-45bd-bfd1-a7f275bb6b77 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:53.458178 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a08a40-1ccd-45bd-bfd1-a7f275bb6b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:53.458449 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a08a40-1ccd-45bd-bfd1-a7f275bb6b77 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:53.459113 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a08a40-1ccd-45bd-bfd1-a7f275bb6b77 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:53.459415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a08a40-1ccd-45bd-bfd1-a7f275bb6b77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:53.460778 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a08a40-1ccd-45bd-bfd1-a7f275bb6b77 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:53.470015 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 499/1001] 10.222.0.74 () {56 vars in 1112 bytes} [Wed Jul 27 16:23:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:53.471101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73da89f-3f19-45d7-b4e9-d504883eed28 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:53.555130 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:53.574070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73da89f-3f19-45d7-b4e9-d504883eed28 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:53.608008 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd736d-3407-4fbb-b62e-5cb7cdc841b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:53.608624 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd736d-3407-4fbb-b62e-5cb7cdc841b7 None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:53.608913 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd736d-3407-4fbb-b62e-5cb7cdc841b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:53.609200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd736d-3407-4fbb-b62e-5cb7cdc841b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:53.609954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd736d-3407-4fbb-b62e-5cb7cdc841b7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:53.611361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd736d-3407-4fbb-b62e-5cb7cdc841b7 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:53.635617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 503/1002] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:23:53 2022] POST /identity/v3/endpoints => generated 334 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:23:53.739993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99e881-fc1c-4713-ba53-b7591302bc7f None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:53.740335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99e881-fc1c-4713-ba53-b7591302bc7f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:53.740566 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99e881-fc1c-4713-ba53-b7591302bc7f None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:53.742140 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 500/1003] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:53.747659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c783d4-3479-4d3f-b8e8-5690ca85dde6 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:53.747969 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c783d4-3479-4d3f-b8e8-5690ca85dde6 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:53.748231 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c783d4-3479-4d3f-b8e8-5690ca85dde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:53.767505 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-56c783d4-3479-4d3f-b8e8-5690ca85dde6 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:53.768768 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-56c783d4-3479-4d3f-b8e8-5690ca85dde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:53.847207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 504/1004] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:53 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:53.868743 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99e881-fc1c-4713-ba53-b7591302bc7f None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:53.974096 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99e881-fc1c-4713-ba53-b7591302bc7f None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:54.009284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611064e-b886-49a1-a1d9-eab9076efd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:54.009949 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611064e-b886-49a1-a1d9-eab9076efd67 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:54.010685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611064e-b886-49a1-a1d9-eab9076efd67 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:54.010961 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611064e-b886-49a1-a1d9-eab9076efd67 None admin] PATH_INFO: `/v3/services/image` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:54.011587 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611064e-b886-49a1-a1d9-eab9076efd67 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:54.012979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611064e-b886-49a1-a1d9-eab9076efd67 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:54.021865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-7611064e-b886-49a1-a1d9-eab9076efd67 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 16:23:54.021865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:54.021865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:23:54.021865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:23:54.021865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:23:54.021865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:23:54.021865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:23:54.021865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:54.021865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 16:23:54.021865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:54.021865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:54.021865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:54.021865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:54.021865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:54.021865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:54.021865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:54.021865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:54.023037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:54.023037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:54.023037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:54.023037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:54.023037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:54.023037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:54.023037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:23:54.023037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:23:54.023037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:23:54.023037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:23:54.023037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:54.023037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:54.023037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:54.023037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:54.023037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:54.023037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:54.023884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:54.023884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:54.023884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:54.023884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:54.023884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:54.023884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:54.023884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:54.023884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:54.023884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:54.023884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:54.023884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:23:54.023884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:54.023884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 16:23:54.023884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:54.023884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 501/1005] 10.222.0.74 () {56 vars in 1105 bytes} [Wed Jul 27 16:23:53 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:54.029005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c783d4-3479-4d3f-b8e8-5690ca85dde6 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:54.093975 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:54.130384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c783d4-3479-4d3f-b8e8-5690ca85dde6 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:54.171320 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816d0ef-4332-4f6e-84b3-e852315f7e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:54.171965 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816d0ef-4332-4f6e-84b3-e852315f7e85 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:54.172233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816d0ef-4332-4f6e-84b3-e852315f7e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:54.172510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816d0ef-4332-4f6e-84b3-e852315f7e85 None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:54.173226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a816d0ef-4332-4f6e-84b3-e852315f7e85 None admin] RBAC: Adding query filter params (name=image) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:54.173538 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a816d0ef-4332-4f6e-84b3-e852315f7e85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:54.174914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a816d0ef-4332-4f6e-84b3-e852315f7e85 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:54.179903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 505/1006] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:54.186061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611064e-b886-49a1-a1d9-eab9076efd67 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:54.199126 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:54.290854 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611064e-b886-49a1-a1d9-eab9076efd67 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:54.324925 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093373b-43d1-4956-a64c-70c516992538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:54.325784 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093373b-43d1-4956-a64c-70c516992538 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:54.326077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093373b-43d1-4956-a64c-70c516992538 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:54.326355 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093373b-43d1-4956-a64c-70c516992538 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:54.327039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093373b-43d1-4956-a64c-70c516992538 None admin] RBAC: Adding query filter params (type=image) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:54.327345 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093373b-43d1-4956-a64c-70c516992538 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:54.328722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093373b-43d1-4956-a64c-70c516992538 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:54.333393 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 502/1007] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:54 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:54.889970 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82065831-8e81-4f40-82cf-6d0115a6c9e6 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:54.890375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82065831-8e81-4f40-82cf-6d0115a6c9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:54.890555 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82065831-8e81-4f40-82cf-6d0115a6c9e6 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:54.891896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 506/1008] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:54.897340 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50282d9a-5be8-4a4f-b26e-e3e2aec3331b None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:54.897632 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50282d9a-5be8-4a4f-b26e-e3e2aec3331b None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:54.897906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50282d9a-5be8-4a4f-b26e-e3e2aec3331b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:54.916128 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-50282d9a-5be8-4a4f-b26e-e3e2aec3331b None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:54.917926 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-50282d9a-5be8-4a4f-b26e-e3e2aec3331b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:54.984087 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 503/1009] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:54 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:55.007260 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82065831-8e81-4f40-82cf-6d0115a6c9e6 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:55.110751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82065831-8e81-4f40-82cf-6d0115a6c9e6 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:55.139369 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:55.146100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bdf8d8-478a-4ceb-8edf-35504a3c354d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:55.146681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdf8d8-478a-4ceb-8edf-35504a3c354d None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:55.146961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdf8d8-478a-4ceb-8edf-35504a3c354d None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:55.147233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdf8d8-478a-4ceb-8edf-35504a3c354d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:55.147872 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bdf8d8-478a-4ceb-8edf-35504a3c354d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:55.152262 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bdf8d8-478a-4ceb-8edf-35504a3c354d None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:55.162028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-b3bdf8d8-478a-4ceb-8edf-35504a3c354d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 16:23:55.162028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:55.162028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:23:55.162028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:23:55.162028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:23:55.162028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:23:55.162028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:23:55.162028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:55.162028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 16:23:55.162028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:55.162028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:55.162028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:55.162028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:55.162028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:55.162028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:55.162028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:55.162028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:55.163052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:55.163052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:55.163052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:55.163052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:55.163052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:55.163052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:55.163052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:23:55.163052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:23:55.163052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:23:55.163052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:23:55.163052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:55.163052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:55.163052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:55.163052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:55.163052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:55.163052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:55.169129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:55.169129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:55.169129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:55.169129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:55.169129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:55.169129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:55.169129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:55.169129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:55.169129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:55.169129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:55.169129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:23:55.169129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:55.169129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 16:23:55.169129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:55.169129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 507/1010] 10.222.0.74 () {56 vars in 1126 bytes} [Wed Jul 27 16:23:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:55.171720 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50282d9a-5be8-4a4f-b26e-e3e2aec3331b None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:55.177117 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:55.276704 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50282d9a-5be8-4a4f-b26e-e3e2aec3331b None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:55.310351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76084ae5-c8cb-41fe-967b-e2f2d36b8673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:55.311203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76084ae5-c8cb-41fe-967b-e2f2d36b8673 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:55.311631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76084ae5-c8cb-41fe-967b-e2f2d36b8673 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:55.312076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76084ae5-c8cb-41fe-967b-e2f2d36b8673 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:55.312926 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76084ae5-c8cb-41fe-967b-e2f2d36b8673 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:55.313398 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76084ae5-c8cb-41fe-967b-e2f2d36b8673 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:55.314908 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76084ae5-c8cb-41fe-967b-e2f2d36b8673 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:55.324708 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 504/1011] 10.222.0.74 () {56 vars in 1122 bytes} [Wed Jul 27 16:23:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:55.326587 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bdf8d8-478a-4ceb-8edf-35504a3c354d None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:55.431485 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bdf8d8-478a-4ceb-8edf-35504a3c354d None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:55.468420 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54933377-4b8d-4ecb-a0e4-8616cdee661d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:55.469069 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54933377-4b8d-4ecb-a0e4-8616cdee661d None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:55.469353 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54933377-4b8d-4ecb-a0e4-8616cdee661d None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:55.469629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54933377-4b8d-4ecb-a0e4-8616cdee661d None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:55.470299 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54933377-4b8d-4ecb-a0e4-8616cdee661d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:55.470604 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54933377-4b8d-4ecb-a0e4-8616cdee661d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:55.471976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54933377-4b8d-4ecb-a0e4-8616cdee661d None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:55.477366 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 508/1012] 10.222.0.74 () {56 vars in 1122 bytes} [Wed Jul 27 16:23:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:55.483015 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76084ae5-c8cb-41fe-967b-e2f2d36b8673 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:55.550206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:55.584294 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76084ae5-c8cb-41fe-967b-e2f2d36b8673 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:55.618973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d650f-c1c7-44ed-9ce1-c4dfc15149e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:55.619618 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d650f-c1c7-44ed-9ce1-c4dfc15149e5 None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:55.619901 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d650f-c1c7-44ed-9ce1-c4dfc15149e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:55.620176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d650f-c1c7-44ed-9ce1-c4dfc15149e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:55.620905 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d650f-c1c7-44ed-9ce1-c4dfc15149e5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:55.622281 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d650f-c1c7-44ed-9ce1-c4dfc15149e5 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:55.638344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 505/1013] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:23:55 2022] POST /identity/v3/endpoints => generated 325 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:23:55.793335 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6a25e-4a30-4ac5-a41b-820a5a366a72 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:55.793620 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6a25e-4a30-4ac5-a41b-820a5a366a72 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:55.793902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6a25e-4a30-4ac5-a41b-820a5a366a72 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:55.795241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 509/1014] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:55.800696 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03020ac6-7a05-43e7-b76a-341bb500115c None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:55.801019 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03020ac6-7a05-43e7-b76a-341bb500115c None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:55.801298 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03020ac6-7a05-43e7-b76a-341bb500115c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:55.820087 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-03020ac6-7a05-43e7-b76a-341bb500115c None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:55.821301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-03020ac6-7a05-43e7-b76a-341bb500115c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:55.896076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 506/1015] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:55 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:55.919236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6a25e-4a30-4ac5-a41b-820a5a366a72 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:55.979757 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:56.027511 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6a25e-4a30-4ac5-a41b-820a5a366a72 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:56.061894 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716d9b2-4af3-4cfc-8cf4-681a1e524302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:56.062458 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716d9b2-4af3-4cfc-8cf4-681a1e524302 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:56.062737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716d9b2-4af3-4cfc-8cf4-681a1e524302 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:56.063013 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716d9b2-4af3-4cfc-8cf4-681a1e524302 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:56.073558 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3716d9b2-4af3-4cfc-8cf4-681a1e524302 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:23:56.078737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-3716d9b2-4af3-4cfc-8cf4-681a1e524302 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 16:23:56.078737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:56.078737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:56.078737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:56.078737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:56.078737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:56.078737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:56.078737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:56.078737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:56.078737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:56.078737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:56.078737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:56.078737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:23:56.078737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:23:56.078737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:23:56.078737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:23:56.079655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:23:56.079655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:23:56.079655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:23:56.079655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:23:56.079655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:56.079655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:56.079655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:56.079655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:56.079655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:56.079655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:56.079655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:56.079655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:56.079655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:56.079655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:56.079655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:56.079655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:56.080488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:56.080488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:56.080488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:56.080488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:56.080488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:23:56.080488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:23:56.080488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:23:56.080488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:23:56.080488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:23:56.080488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:23:56.080488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 16:23:56.080488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:56.084748 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 510/1016] 10.222.0.74 () {56 vars in 1120 bytes} [Wed Jul 27 16:23:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:56.086175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03020ac6-7a05-43e7-b76a-341bb500115c None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:56.144329 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:56.193743 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03020ac6-7a05-43e7-b76a-341bb500115c None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:56.229234 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10c20f-1489-4afa-bf86-213d5eeb481a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:56.229850 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10c20f-1489-4afa-bf86-213d5eeb481a None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:56.230130 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10c20f-1489-4afa-bf86-213d5eeb481a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:56.230399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10c20f-1489-4afa-bf86-213d5eeb481a None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:56.231105 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10c20f-1489-4afa-bf86-213d5eeb481a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:56.231407 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10c20f-1489-4afa-bf86-213d5eeb481a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:56.232776 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10c20f-1489-4afa-bf86-213d5eeb481a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:56.241011 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 507/1017] 10.222.0.74 () {56 vars in 1115 bytes} [Wed Jul 27 16:23:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:56.247779 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716d9b2-4af3-4cfc-8cf4-681a1e524302 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:56.340398 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32073bd8-6126-4aa5-bb2c-ddd74fe3df72 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:56.340659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32073bd8-6126-4aa5-bb2c-ddd74fe3df72 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:56.340906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32073bd8-6126-4aa5-bb2c-ddd74fe3df72 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:56.345405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 508/1018] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:56.349406 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686ddca-b2c6-456e-9b54-a905e4c1142c None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:56.349869 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686ddca-b2c6-456e-9b54-a905e4c1142c None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:56.350303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686ddca-b2c6-456e-9b54-a905e4c1142c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:56.358891 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716d9b2-4af3-4cfc-8cf4-681a1e524302 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:56.371283 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-b686ddca-b2c6-456e-9b54-a905e4c1142c None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:56.373394 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b686ddca-b2c6-456e-9b54-a905e4c1142c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:56.378342 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:56.400821 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474615ee-af45-4c37-af13-f4b19ec8a872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:56.401502 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474615ee-af45-4c37-af13-f4b19ec8a872 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:56.401790 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474615ee-af45-4c37-af13-f4b19ec8a872 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:56.402070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474615ee-af45-4c37-af13-f4b19ec8a872 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:56.413213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474615ee-af45-4c37-af13-f4b19ec8a872 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:56.414673 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474615ee-af45-4c37-af13-f4b19ec8a872 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:56.429043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-474615ee-af45-4c37-af13-f4b19ec8a872 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:56.429043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:56.429043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:56.429043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:56.429043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:56.429043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:56.429043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:56.429043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:56.429043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:56.429043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:56.429043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:56.429043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:56.429043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:56.429043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:56.429043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:56.429951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:56.429951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:56.429951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:56.429951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:56.429951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:56.429951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:56.429951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:56.429951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:56.429951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:56.429951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:56.429951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:56.429951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:56.429951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:56.429951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:56.429951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:56.429951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:56.429951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:56.430837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:56.430837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:56.430837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:56.430837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:56.430837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 511/1019] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:56.436108 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474615ee-af45-4c37-af13-f4b19ec8a872 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:56.456921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 509/1020] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:56 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:56.478597 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686ddca-b2c6-456e-9b54-a905e4c1142c None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:56.534998 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:56.539469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474615ee-af45-4c37-af13-f4b19ec8a872 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:56.580587 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d371ccca-88db-47a6-86de-6e072b5caf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:56.581229 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371ccca-88db-47a6-86de-6e072b5caf8e None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:56.581508 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371ccca-88db-47a6-86de-6e072b5caf8e None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:56.581779 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371ccca-88db-47a6-86de-6e072b5caf8e None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:56.582453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d371ccca-88db-47a6-86de-6e072b5caf8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:56.582757 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d371ccca-88db-47a6-86de-6e072b5caf8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:56.584115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d371ccca-88db-47a6-86de-6e072b5caf8e None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:56.591102 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686ddca-b2c6-456e-9b54-a905e4c1142c None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:56.597418 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 512/1021] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:56.602700 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d371ccca-88db-47a6-86de-6e072b5caf8e None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:56.628821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ff9d0-08f7-4e81-9297-245ba028a597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:56.629472 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ff9d0-08f7-4e81-9297-245ba028a597 None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:56.629770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ff9d0-08f7-4e81-9297-245ba028a597 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:56.630043 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ff9d0-08f7-4e81-9297-245ba028a597 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:56.630724 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ff9d0-08f7-4e81-9297-245ba028a597 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:56.632091 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ff9d0-08f7-4e81-9297-245ba028a597 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:56.640193 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:56.653462 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 510/1022] 10.222.0.74 () {60 vars in 1141 bytes} [Wed Jul 27 16:23:56 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:23:56.722810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d371ccca-88db-47a6-86de-6e072b5caf8e None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:56.770575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aad7d-17d7-4163-a6dc-301cf60dbb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:56.771265 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aad7d-17d7-4163-a6dc-301cf60dbb98 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:56.771544 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aad7d-17d7-4163-a6dc-301cf60dbb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:56.771891 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aad7d-17d7-4163-a6dc-301cf60dbb98 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:56.780681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435aad7d-17d7-4163-a6dc-301cf60dbb98 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:56.782240 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435aad7d-17d7-4163-a6dc-301cf60dbb98 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:56.795860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-435aad7d-17d7-4163-a6dc-301cf60dbb98 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 16:23:56.795860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:56.795860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:56.795860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:56.795860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:56.795860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:56.795860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:56.795860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:56.795860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:56.795860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:56.795860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:56.795860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:56.795860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:23:56.795860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:23:56.795860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:23:56.795860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:23:56.796930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:56.796930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:56.796930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:23:56.796930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:56.796930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:23:56.796930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:23:56.796930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:56.796930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:56.796930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:56.796930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:56.796930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:56.796930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:56.796930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:56.796930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:56.796930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:56.796930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:56.796930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:56.796930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:56.797883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:56.797883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:56.797883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:23:56.797883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:23:56.797883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:23:56.797883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:23:56.797883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:23:56.797883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:23:56.797883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 16:23:56.797883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:23:56.797883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 513/1023] 10.222.0.74 () {56 vars in 1093 bytes} [Wed Jul 27 16:23:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:23:56.802777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ff9d0-08f7-4e81-9297-245ba028a597 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:56.907756 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ff9d0-08f7-4e81-9297-245ba028a597 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:56.914660 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:56.943310 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ff093-966c-4000-97cd-360109434d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:56.944012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ff093-966c-4000-97cd-360109434d61 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:56.944399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ff093-966c-4000-97cd-360109434d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:56.944737 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ff093-966c-4000-97cd-360109434d61 None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:56.945616 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ff093-966c-4000-97cd-360109434d61 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:56.945987 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ff093-966c-4000-97cd-360109434d61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:56.947992 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ff093-966c-4000-97cd-360109434d61 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:56.965837 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 511/1024] 10.222.0.74 () {56 vars in 1133 bytes} [Wed Jul 27 16:23:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:56.971353 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aad7d-17d7-4163-a6dc-301cf60dbb98 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:57.057381 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:57.073102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aad7d-17d7-4163-a6dc-301cf60dbb98 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:57.107312 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf63be-2f18-4fae-b4ff-5b91738fcf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:57.108017 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf63be-2f18-4fae-b4ff-5b91738fcf00 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:57.108364 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf63be-2f18-4fae-b4ff-5b91738fcf00 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:57.108701 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf63be-2f18-4fae-b4ff-5b91738fcf00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:57.183149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf63be-2f18-4fae-b4ff-5b91738fcf00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:57.184592 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf63be-2f18-4fae-b4ff-5b91738fcf00 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:57.268800 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:57.291136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 514/1025] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:23:56 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 16:23:57.293293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ff093-966c-4000-97cd-360109434d61 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:57.392023 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ff093-966c-4000-97cd-360109434d61 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:57.425608 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4dfc5-b9e3-4759-8c81-0ac28252d0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:57.426170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4dfc5-b9e3-4759-8c81-0ac28252d0ab None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:57.426443 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4dfc5-b9e3-4759-8c81-0ac28252d0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:57.426707 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4dfc5-b9e3-4759-8c81-0ac28252d0ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:57.436713 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4dfc5-b9e3-4759-8c81-0ac28252d0ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:57.438185 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4dfc5-b9e3-4759-8c81-0ac28252d0ab None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:57.453181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-cef4dfc5-b9e3-4759-8c81-0ac28252d0ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:57.453181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:57.453181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:57.453181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:57.453181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:57.453181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:57.453181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:57.453181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:57.453181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:57.453181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:57.453181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:57.453181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:57.453181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:23:57.453181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:23:57.453181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:23:57.454080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:23:57.454080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:57.454080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:57.454080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:57.454080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:57.454080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:57.454080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:57.454080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:57.454080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:57.454080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:57.454080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:57.454080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:57.454080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:57.454080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:57.454080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:57.454080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:57.454080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:57.454970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:23:57.454970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:23:57.454970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:23:57.454970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:57.454970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 512/1026] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:23:57.461072 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf63be-2f18-4fae-b4ff-5b91738fcf00 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:57.561538 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf63be-2f18-4fae-b4ff-5b91738fcf00 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:57.595588 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd37d8-ca2c-481f-a2de-87630eb1d4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:57.596212 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd37d8-ca2c-481f-a2de-87630eb1d4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:57.596494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd37d8-ca2c-481f-a2de-87630eb1d4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:57.596769 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd37d8-ca2c-481f-a2de-87630eb1d4e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:57.597509 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd37d8-ca2c-481f-a2de-87630eb1d4e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:57.597800 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd37d8-ca2c-481f-a2de-87630eb1d4e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:57.599160 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd37d8-ca2c-481f-a2de-87630eb1d4e5 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:57.615390 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 515/1027] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:57.616855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4dfc5-b9e3-4759-8c81-0ac28252d0ab None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:57.637238 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:57.692939 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359198c-8908-4a51-a5b1-8b7a649e3ea1 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:57.693241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359198c-8908-4a51-a5b1-8b7a649e3ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:57.693445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359198c-8908-4a51-a5b1-8b7a649e3ea1 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:57.694959 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 516/1028] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:23:57.699689 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e3980-764c-447d-9339-fd43a1aa50d3 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:57.699980 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e3980-764c-447d-9339-fd43a1aa50d3 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:57.700261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e3980-764c-447d-9339-fd43a1aa50d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:57.721760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-d86e3980-764c-447d-9339-fd43a1aa50d3 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:57.722963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4dfc5-b9e3-4759-8c81-0ac28252d0ab None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:57.723300 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d86e3980-764c-447d-9339-fd43a1aa50d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:57.760143 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbe6f5-b53e-4211-9d6a-e3aae0dfc731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:57.760783 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbe6f5-b53e-4211-9d6a-e3aae0dfc731 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:57.761199 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbe6f5-b53e-4211-9d6a-e3aae0dfc731 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:57.761477 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbe6f5-b53e-4211-9d6a-e3aae0dfc731 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:57.768826 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbe6f5-b53e-4211-9d6a-e3aae0dfc731 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:57.770371 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbe6f5-b53e-4211-9d6a-e3aae0dfc731 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:57.772665 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:57.783348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-b0cbe6f5-b53e-4211-9d6a-e3aae0dfc731 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:57.783348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:57.783348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:57.783348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:57.783348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:57.783348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:57.783348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:57.783348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:57.783348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:57.783348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:57.783348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:57.783348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:57.783348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:23:57.783348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:23:57.783348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:23:57.784396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:23:57.784396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:57.784396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:57.784396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:57.784396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:57.784396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:57.784396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:57.784396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:57.784396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:57.784396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:57.784396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:57.784396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:57.784396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:57.784396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:57.784396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:57.784396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:57.784396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:57.785331 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:23:57.785331 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:23:57.785331 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:23:57.785331 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:23:57.785331 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:23:57.785331 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:23:57.785331 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:23:57.785331 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:57.789494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 513/1029] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:23:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:23:57.790431 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbe6f5-b53e-4211-9d6a-e3aae0dfc731 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:57.806008 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 517/1030] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 16:23:57.829568 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e3980-764c-447d-9339-fd43a1aa50d3 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:57.873159 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:57.900035 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbe6f5-b53e-4211-9d6a-e3aae0dfc731 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:57.930273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e3980-764c-447d-9339-fd43a1aa50d3 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:57.935579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe291e5-d3f6-424b-afad-7379c41b4ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:57.936201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe291e5-d3f6-424b-afad-7379c41b4ddb None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:57.936485 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe291e5-d3f6-424b-afad-7379c41b4ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:57.936760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe291e5-d3f6-424b-afad-7379c41b4ddb None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:57.937533 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe291e5-d3f6-424b-afad-7379c41b4ddb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:57.937843 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe291e5-d3f6-424b-afad-7379c41b4ddb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:57.939279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe291e5-d3f6-424b-afad-7379c41b4ddb None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:57.952168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 514/1031] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:23:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:57.957960 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe291e5-d3f6-424b-afad-7379c41b4ddb None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:57.969123 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e3341-7a5e-4b2f-a753-0023e00c261c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:57.970078 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e3341-7a5e-4b2f-a753-0023e00c261c None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:57.970368 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e3341-7a5e-4b2f-a753-0023e00c261c None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:57.970799 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e3341-7a5e-4b2f-a753-0023e00c261c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:57.971991 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e3341-7a5e-4b2f-a753-0023e00c261c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:57.973428 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e3341-7a5e-4b2f-a753-0023e00c261c None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:57.977063 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 518/1032] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:57.981188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e3341-7a5e-4b2f-a753-0023e00c261c None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:58.032831 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:58.063022 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe291e5-d3f6-424b-afad-7379c41b4ddb None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:58.090371 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e3341-7a5e-4b2f-a753-0023e00c261c None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:58.099336 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a957df9-6b17-42a8-95b0-9bf8315ee77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:58.100115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a957df9-6b17-42a8-95b0-9bf8315ee77e None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:58.100237 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a957df9-6b17-42a8-95b0-9bf8315ee77e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:58.100495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a957df9-6b17-42a8-95b0-9bf8315ee77e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:58.101309 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a957df9-6b17-42a8-95b0-9bf8315ee77e None admin] RBAC: Adding query filter params (role.id=2bafbadedabe4d7681043ae2e7cadacd, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=56dfafd15e284b7986ba07dd3819bf65) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:58.101866 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a957df9-6b17-42a8-95b0-9bf8315ee77e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:58.103053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a957df9-6b17-42a8-95b0-9bf8315ee77e None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:58.106375 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:58.110426 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 515/1033] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:23:57 2022] GET /identity/v3/role_assignments?user.id=56dfafd15e284b7986ba07dd3819bf65&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=2bafbadedabe4d7681043ae2e7cadacd => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:58.136447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08943c24-0541-49ee-883e-74a4c2d3a463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:58.137126 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08943c24-0541-49ee-883e-74a4c2d3a463 None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:58.137474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08943c24-0541-49ee-883e-74a4c2d3a463 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:58.137753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08943c24-0541-49ee-883e-74a4c2d3a463 None admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:58.138482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08943c24-0541-49ee-883e-74a4c2d3a463 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:58.139955 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08943c24-0541-49ee-883e-74a4c2d3a463 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:58.180923 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 519/1034] 10.222.0.74 () {60 vars in 1141 bytes} [Wed Jul 27 16:23:57 2022] POST /identity/v3/projects => generated 311 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:23:58.687752 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c02ddb-8315-4457-991e-15b6aee0a800 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:58.688260 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c02ddb-8315-4457-991e-15b6aee0a800 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:58.688368 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c02ddb-8315-4457-991e-15b6aee0a800 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:58.689993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 516/1035] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:23:58.695122 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d707a6-3c4f-44ee-a3b5-7322cc891c9c None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:58.695534 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d707a6-3c4f-44ee-a3b5-7322cc891c9c None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:58.695871 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d707a6-3c4f-44ee-a3b5-7322cc891c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:58.714670 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-c0d707a6-3c4f-44ee-a3b5-7322cc891c9c None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:23:58.715833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-c0d707a6-3c4f-44ee-a3b5-7322cc891c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:23:58.785189 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 520/1036] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:23:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:23:58.807191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c02ddb-8315-4457-991e-15b6aee0a800 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:58.909629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c02ddb-8315-4457-991e-15b6aee0a800 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:58.943741 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e90c62-a38c-44d4-9d2d-97d9ff478eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:58.944307 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e90c62-a38c-44d4-9d2d-97d9ff478eda None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:58.944583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e90c62-a38c-44d4-9d2d-97d9ff478eda None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:58.944851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e90c62-a38c-44d4-9d2d-97d9ff478eda None admin] PATH_INFO: `/v3/services/image` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:58.945493 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e90c62-a38c-44d4-9d2d-97d9ff478eda None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:58.946853 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e90c62-a38c-44d4-9d2d-97d9ff478eda None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:58.955895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-75e90c62-a38c-44d4-9d2d-97d9ff478eda None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 16:23:58.955895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:58.955895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:23:58.955895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:23:58.955895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:23:58.955895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:23:58.955895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:23:58.955895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:58.955895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 16:23:58.955895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:58.955895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:23:58.955895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:58.955895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:23:58.955895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:23:58.955895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:23:58.955895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:23:58.955895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:23:58.956864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:23:58.956864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:23:58.956864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:23:58.956864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:23:58.956864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:23:58.956864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:23:58.956864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:23:58.956864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:23:58.956864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:23:58.956864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:23:58.956864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:23:58.956864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:23:58.956864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:23:58.956864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:23:58.956864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:23:58.956864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:23:58.957748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:23:58.957748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:23:58.957748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:23:58.957748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:23:58.957748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:23:58.957748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:23:58.957748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:23:58.957748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:23:58.957748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:23:58.957748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:23:58.957748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:23:58.957748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:23:58.957748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 16:23:58.957748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:23:58.961940 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 517/1037] 10.222.0.74 () {56 vars in 1105 bytes} [Wed Jul 27 16:23:58 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:23:58.963191 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d707a6-3c4f-44ee-a3b5-7322cc891c9c None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:58.989968 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:59.063290 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d707a6-3c4f-44ee-a3b5-7322cc891c9c None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:59.097351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5100c6-2db5-48c6-a2e9-d305b94805a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:59.097956 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5100c6-2db5-48c6-a2e9-d305b94805a6 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:59.098233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5100c6-2db5-48c6-a2e9-d305b94805a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:59.098507 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5100c6-2db5-48c6-a2e9-d305b94805a6 None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:59.099167 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5100c6-2db5-48c6-a2e9-d305b94805a6 None admin] RBAC: Adding query filter params (name=image) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:59.099465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5100c6-2db5-48c6-a2e9-d305b94805a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:59.100815 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5100c6-2db5-48c6-a2e9-d305b94805a6 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:59.105759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 521/1038] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:59.111363 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e90c62-a38c-44d4-9d2d-97d9ff478eda None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:59.182538 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:59.224044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e90c62-a38c-44d4-9d2d-97d9ff478eda None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:59.258050 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925e8e4-2cd9-4d51-99e6-8a5bcf832e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:59.258673 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925e8e4-2cd9-4d51-99e6-8a5bcf832e41 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:59.258958 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925e8e4-2cd9-4d51-99e6-8a5bcf832e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:59.259235 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925e8e4-2cd9-4d51-99e6-8a5bcf832e41 None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:59.260562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925e8e4-2cd9-4d51-99e6-8a5bcf832e41 None admin] RBAC: Adding query filter params (type=image) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:59.260870 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925e8e4-2cd9-4d51-99e6-8a5bcf832e41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:59.262273 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925e8e4-2cd9-4d51-99e6-8a5bcf832e41 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:59.271839 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 518/1039] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:23:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:23:59.273076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5100c6-2db5-48c6-a2e9-d305b94805a6 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:59.365278 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:59.376678 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5100c6-2db5-48c6-a2e9-d305b94805a6 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:59.413452 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca11fb-daa7-4ab9-b6a4-771f709ef0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:59.414072 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca11fb-daa7-4ab9-b6a4-771f709ef0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:59.414354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca11fb-daa7-4ab9-b6a4-771f709ef0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:59.414629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca11fb-daa7-4ab9-b6a4-771f709ef0e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:59.415378 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca11fb-daa7-4ab9-b6a4-771f709ef0e6 None admin] RBAC: Adding query filter params (interface=public, service_id=b00db22cedc04f4fa5845f53ccf30413, region_id=RegionOne) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:23:59.415690 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca11fb-daa7-4ab9-b6a4-771f709ef0e6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:59.417109 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca11fb-daa7-4ab9-b6a4-771f709ef0e6 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:59.426388 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 522/1040] 10.222.0.74 () {56 vars in 1251 bytes} [Wed Jul 27 16:23:59 2022] GET /identity/v3/endpoints?service_id=b00db22cedc04f4fa5845f53ccf30413&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:23:59.427534 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925e8e4-2cd9-4d51-99e6-8a5bcf832e41 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:23:59.526951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925e8e4-2cd9-4d51-99e6-8a5bcf832e41 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:23:59.552348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:23:59.562047 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc755e-982d-43c5-bb93-06550677c43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:23:59.562665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc755e-982d-43c5-bb93-06550677c43a None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:23:59.562950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc755e-982d-43c5-bb93-06550677c43a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:23:59.563228 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc755e-982d-43c5-bb93-06550677c43a None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:23:59.563882 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc755e-982d-43c5-bb93-06550677c43a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:23:59.565294 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc755e-982d-43c5-bb93-06550677c43a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:23:59.580920 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 519/1041] 10.222.0.74 () {56 vars in 1087 bytes} [Wed Jul 27 16:23:59 2022] GET /identity/v3/services => generated 1957 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:24:00.096403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5ec81-eecb-4cad-a54a-79bc3a4c7deb None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:00.096951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5ec81-eecb-4cad-a54a-79bc3a4c7deb None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:00.096951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5ec81-eecb-4cad-a54a-79bc3a4c7deb None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:00.098451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 523/1042] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:24:00.103480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b098ae3-2c09-447e-9f09-edffb31575ba None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:00.103864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b098ae3-2c09-447e-9f09-edffb31575ba None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:00.104032 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b098ae3-2c09-447e-9f09-edffb31575ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:00.122415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-3b098ae3-2c09-447e-9f09-edffb31575ba None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:00.123595 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-3b098ae3-2c09-447e-9f09-edffb31575ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:00.198815 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 520/1043] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:24:00.207190 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3839b-a878-4a14-9265-9cc767423a05 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:00.207477 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3839b-a878-4a14-9265-9cc767423a05 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:00.207754 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3839b-a878-4a14-9265-9cc767423a05 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:00.212640 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 524/1044] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:24:00.214233 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b6dd6-e2c4-47ba-834f-99828abc2193 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:00.214523 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b6dd6-e2c4-47ba-834f-99828abc2193 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:00.214805 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b6dd6-e2c4-47ba-834f-99828abc2193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:00.222544 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3839b-a878-4a14-9265-9cc767423a05 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:00.232670 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-c30b6dd6-e2c4-47ba-834f-99828abc2193 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:00.233874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c30b6dd6-e2c4-47ba-834f-99828abc2193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:00.302088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 521/1045] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 16:24:00.326580 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b6dd6-e2c4-47ba-834f-99828abc2193 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:00.327483 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3839b-a878-4a14-9265-9cc767423a05 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:00.361775 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486544bf-422b-4b8c-96d7-ca99497d69b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:00.362339 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486544bf-422b-4b8c-96d7-ca99497d69b4 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:00.362617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486544bf-422b-4b8c-96d7-ca99497d69b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:00.362887 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486544bf-422b-4b8c-96d7-ca99497d69b4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:00.373491 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486544bf-422b-4b8c-96d7-ca99497d69b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:24:00.378403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-486544bf-422b-4b8c-96d7-ca99497d69b4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 16:24:00.378403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:00.378403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:24:00.378403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:24:00.378403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:24:00.378403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:24:00.378403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:24:00.378403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:24:00.378403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:24:00.378403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:24:00.378403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:24:00.378403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:24:00.378403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:24:00.378403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:24:00.378403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:24:00.378403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:24:00.379360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:24:00.379360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:24:00.379360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:24:00.379360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:24:00.379360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:24:00.379360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:24:00.379360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:24:00.379360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:24:00.379360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:24:00.379360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:24:00.379360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:24:00.379360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:24:00.379360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:24:00.379360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:24:00.379360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:24:00.379360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:24:00.380266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:24:00.380266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:24:00.380266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:24:00.380266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:24:00.380266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:24:00.380266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:24:00.380266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:24:00.380266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:24:00.380266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:24:00.380266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:24:00.380266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 16:24:00.380266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:24:00.381329 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:00.384600 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 525/1046] 10.222.0.74 () {56 vars in 1120 bytes} [Wed Jul 27 16:24:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:24:00.386002 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486544bf-422b-4b8c-96d7-ca99497d69b4 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:00.402475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:00.430687 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b6dd6-e2c4-47ba-834f-99828abc2193 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:00.467529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf09f0f-a3dc-4854-ab4c-3e8d73b4e348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:00.468193 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf09f0f-a3dc-4854-ab4c-3e8d73b4e348 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:00.468477 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf09f0f-a3dc-4854-ab4c-3e8d73b4e348 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:00.468749 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf09f0f-a3dc-4854-ab4c-3e8d73b4e348 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:00.469941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf09f0f-a3dc-4854-ab4c-3e8d73b4e348 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:00.471385 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf09f0f-a3dc-4854-ab4c-3e8d73b4e348 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:00.473155 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 522/1047] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:24:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:24:00.478565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf09f0f-a3dc-4854-ab4c-3e8d73b4e348 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:00.501458 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486544bf-422b-4b8c-96d7-ca99497d69b4 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:00.538222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24d35a-cc32-4e2a-b668-0d591dcfaf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:00.538935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24d35a-cc32-4e2a-b668-0d591dcfaf8f None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:00.539221 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24d35a-cc32-4e2a-b668-0d591dcfaf8f None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:00.539497 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24d35a-cc32-4e2a-b668-0d591dcfaf8f None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:00.540207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24d35a-cc32-4e2a-b668-0d591dcfaf8f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:00.540513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24d35a-cc32-4e2a-b668-0d591dcfaf8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:00.541923 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24d35a-cc32-4e2a-b668-0d591dcfaf8f None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:00.550207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 526/1048] 10.222.0.74 () {56 vars in 1115 bytes} [Wed Jul 27 16:24:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:00.555591 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24d35a-cc32-4e2a-b668-0d591dcfaf8f None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:00.584168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf09f0f-a3dc-4854-ab4c-3e8d73b4e348 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:00.605325 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:00.620713 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d2b61-be3b-49b1-810a-d79804fee66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:00.621286 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d2b61-be3b-49b1-810a-d79804fee66d None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:00.621564 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d2b61-be3b-49b1-810a-d79804fee66d None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:00.621835 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d2b61-be3b-49b1-810a-d79804fee66d None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:00.622625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d2b61-be3b-49b1-810a-d79804fee66d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:00.624155 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d2b61-be3b-49b1-810a-d79804fee66d None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:00.634387 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:00.661355 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 523/1049] 10.222.0.74 () {60 vars in 1132 bytes} [Wed Jul 27 16:24:00 2022] POST /identity/v3/users => generated 285 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:24:00.661719 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24d35a-cc32-4e2a-b668-0d591dcfaf8f None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:00.701464 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a69ed3-28d0-442f-a314-eae8f2d5782c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:00.702141 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a69ed3-28d0-442f-a314-eae8f2d5782c None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:00.702425 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a69ed3-28d0-442f-a314-eae8f2d5782c None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:00.702709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a69ed3-28d0-442f-a314-eae8f2d5782c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:00.715018 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a69ed3-28d0-442f-a314-eae8f2d5782c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:00.716522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a69ed3-28d0-442f-a314-eae8f2d5782c None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:00.732954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-11a69ed3-28d0-442f-a314-eae8f2d5782c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:24:00.732954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:00.732954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:24:00.732954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:24:00.732954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:24:00.732954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:24:00.732954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:24:00.732954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:24:00.732954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:24:00.732954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:24:00.732954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:24:00.732954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:24:00.732954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:24:00.732954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:24:00.732954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:24:00.733950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:24:00.733950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:24:00.733950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:24:00.733950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:24:00.733950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:24:00.733950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:24:00.733950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:24:00.733950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:24:00.733950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:24:00.733950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:24:00.733950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:24:00.733950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:24:00.733950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:24:00.733950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:24:00.733950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:24:00.733950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:24:00.733950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:24:00.739753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:24:00.739753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:24:00.739753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:24:00.739753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:24:00.739753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 527/1050] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:24:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:24:00.741724 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d2b61-be3b-49b1-810a-d79804fee66d None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:00.852652 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d2b61-be3b-49b1-810a-d79804fee66d None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:00.891019 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c4744-879a-4ab7-abd4-d97ae2597673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:00.891970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c4744-879a-4ab7-abd4-d97ae2597673 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:00.892248 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c4744-879a-4ab7-abd4-d97ae2597673 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:00.892521 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c4744-879a-4ab7-abd4-d97ae2597673 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:00.893246 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c4744-879a-4ab7-abd4-d97ae2597673 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:00.893554 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c4744-879a-4ab7-abd4-d97ae2597673 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:00.894921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c4744-879a-4ab7-abd4-d97ae2597673 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:00.907643 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 524/1051] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:00.913161 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a69ed3-28d0-442f-a314-eae8f2d5782c None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:01.014413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a69ed3-28d0-442f-a314-eae8f2d5782c None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:01.041782 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:01.050680 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae60aa-d6ea-43a5-8ae0-4de642838d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:01.051323 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae60aa-d6ea-43a5-8ae0-4de642838d37 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:01.051602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae60aa-d6ea-43a5-8ae0-4de642838d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:01.051876 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae60aa-d6ea-43a5-8ae0-4de642838d37 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:01.060010 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae60aa-d6ea-43a5-8ae0-4de642838d37 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:01.061478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae60aa-d6ea-43a5-8ae0-4de642838d37 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:01.072654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-35ae60aa-d6ea-43a5-8ae0-4de642838d37 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 16:24:01.072654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:01.072654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:24:01.072654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:24:01.072654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:24:01.072654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:24:01.072654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:24:01.072654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:24:01.072654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:24:01.072654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:24:01.072654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:24:01.072654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:24:01.072654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:24:01.072654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:24:01.072654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:24:01.072654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:24:01.073759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:24:01.073759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:24:01.073759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:24:01.073759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:24:01.073759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:24:01.073759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:24:01.073759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:24:01.073759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:24:01.073759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:24:01.073759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:24:01.073759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:24:01.073759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:24:01.073759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:24:01.073759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:24:01.073759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:24:01.073759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:24:01.073759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:24:01.073759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:24:01.075289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:24:01.075289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:24:01.075289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:24:01.075289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:24:01.075289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:24:01.075289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:24:01.075289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:24:01.075289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:24:01.075289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 16:24:01.075289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:24:01.076297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 528/1052] 10.222.0.74 () {56 vars in 1093 bytes} [Wed Jul 27 16:24:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:24:01.079996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c4744-879a-4ab7-abd4-d97ae2597673 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:01.123712 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:01.204736 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c4744-879a-4ab7-abd4-d97ae2597673 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:01.239707 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d15369-2f84-4185-be82-bac3cd975ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:01.240332 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d15369-2f84-4185-be82-bac3cd975ccb None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:01.240616 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d15369-2f84-4185-be82-bac3cd975ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:01.240960 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d15369-2f84-4185-be82-bac3cd975ccb None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:01.241850 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d15369-2f84-4185-be82-bac3cd975ccb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:01.242171 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d15369-2f84-4185-be82-bac3cd975ccb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:01.243678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d15369-2f84-4185-be82-bac3cd975ccb None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:01.265934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 525/1053] 10.222.0.74 () {56 vars in 1133 bytes} [Wed Jul 27 16:24:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:01.267131 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae60aa-d6ea-43a5-8ae0-4de642838d37 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:01.371411 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae60aa-d6ea-43a5-8ae0-4de642838d37 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:01.407969 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c26ed9-a99f-4a61-89a2-e13500d48f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:01.408770 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c26ed9-a99f-4a61-89a2-e13500d48f61 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:01.409152 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c26ed9-a99f-4a61-89a2-e13500d48f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:01.409438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c26ed9-a99f-4a61-89a2-e13500d48f61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:01.423033 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c26ed9-a99f-4a61-89a2-e13500d48f61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:01.424435 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c26ed9-a99f-4a61-89a2-e13500d48f61 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:01.440453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-37c26ed9-a99f-4a61-89a2-e13500d48f61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:24:01.440453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:01.440453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:24:01.440453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:24:01.440453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:24:01.440453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:24:01.440453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:24:01.440453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:24:01.440453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:24:01.440453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:24:01.440453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:24:01.440453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:24:01.440453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:24:01.440453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:24:01.440453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:24:01.441586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:24:01.441586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:24:01.441586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:24:01.441586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:24:01.441586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:24:01.441586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:24:01.441586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:24:01.441586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:24:01.441586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:24:01.441586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:24:01.441586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:24:01.441586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:24:01.441586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:24:01.441586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:24:01.441586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:24:01.441586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:24:01.441586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:24:01.442891 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:24:01.442891 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:24:01.442891 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:24:01.442891 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:24:01.443383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 529/1054] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:24:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:24:01.447336 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d15369-2f84-4185-be82-bac3cd975ccb None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:01.543183 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:01.550928 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d15369-2f84-4185-be82-bac3cd975ccb None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:01.584115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c1542-5252-4c89-9863-98b6bb5fa0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:01.584727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c1542-5252-4c89-9863-98b6bb5fa0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:01.585046 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c1542-5252-4c89-9863-98b6bb5fa0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:01.585320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c1542-5252-4c89-9863-98b6bb5fa0b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:01.586005 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c1542-5252-4c89-9863-98b6bb5fa0b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:01.586308 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c1542-5252-4c89-9863-98b6bb5fa0b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:01.587670 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c1542-5252-4c89-9863-98b6bb5fa0b0 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:01.596392 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27e33a-7b03-490f-8d74-8b2be5681823 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:01.597138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27e33a-7b03-490f-8d74-8b2be5681823 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:01.597138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27e33a-7b03-490f-8d74-8b2be5681823 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:01.598562 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 530/1055] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:24:01.600418 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 526/1056] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:01.604116 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3013c27-35f6-4cbf-8cb5-4c6cc1aa727f None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:01.604408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3013c27-35f6-4cbf-8cb5-4c6cc1aa727f None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:01.604700 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3013c27-35f6-4cbf-8cb5-4c6cc1aa727f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:01.605868 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c1542-5252-4c89-9863-98b6bb5fa0b0 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:01.625465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-d3013c27-35f6-4cbf-8cb5-4c6cc1aa727f None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:01.626666 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d3013c27-35f6-4cbf-8cb5-4c6cc1aa727f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:01.664794 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:01.705241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 531/1057] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 16:24:01.721357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c1542-5252-4c89-9863-98b6bb5fa0b0 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:01.724682 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3013c27-35f6-4cbf-8cb5-4c6cc1aa727f None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:01.757179 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2640bee8-be37-4ca9-831a-6077931aa3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:01.757824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640bee8-be37-4ca9-831a-6077931aa3ea None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:01.758103 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640bee8-be37-4ca9-831a-6077931aa3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:01.758379 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640bee8-be37-4ca9-831a-6077931aa3ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:01.765340 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2640bee8-be37-4ca9-831a-6077931aa3ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:01.766795 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2640bee8-be37-4ca9-831a-6077931aa3ea None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:01.778972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-2640bee8-be37-4ca9-831a-6077931aa3ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:24:01.778972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:01.778972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:24:01.778972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:24:01.778972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:24:01.778972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:24:01.778972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:24:01.778972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:24:01.778972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:24:01.778972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:24:01.778972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:24:01.778972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:24:01.778972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:24:01.778972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:24:01.778972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:24:01.779864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:24:01.779864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:24:01.779864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:24:01.779864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:24:01.779864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:24:01.779864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:24:01.779864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:24:01.779864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:24:01.779864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:24:01.779864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:24:01.779864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:24:01.779864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:24:01.779864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:24:01.779864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:24:01.779864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:24:01.779864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:24:01.779864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:24:01.781376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:24:01.781376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:24:01.781376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:24:01.781376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:24:01.781376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:24:01.781376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:24:01.781376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:24:01.781376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:24:01.781376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 527/1058] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:24:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:24:01.785699 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2640bee8-be37-4ca9-831a-6077931aa3ea None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:01.802018 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:01.842620 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3013c27-35f6-4cbf-8cb5-4c6cc1aa727f None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:01.878791 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508798eb-cb27-44be-a2b5-ec0ab5bcf05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:01.879516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508798eb-cb27-44be-a2b5-ec0ab5bcf05d None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:01.879830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508798eb-cb27-44be-a2b5-ec0ab5bcf05d None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:01.880078 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508798eb-cb27-44be-a2b5-ec0ab5bcf05d None admin] PATH_INFO: `/v3/services/image` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:01.880785 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508798eb-cb27-44be-a2b5-ec0ab5bcf05d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:01.882496 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508798eb-cb27-44be-a2b5-ec0ab5bcf05d None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:01.891723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-508798eb-cb27-44be-a2b5-ec0ab5bcf05d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 16:24:01.891723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:01.891723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:24:01.891723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:24:01.891723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:24:01.891723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:24:01.891723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:24:01.891723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:24:01.891723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 16:24:01.891723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:24:01.891723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:24:01.891723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:24:01.891723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:01.891723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:24:01.891723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:24:01.891723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:24:01.891723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:24:01.892733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:24:01.892733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:24:01.892733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:24:01.892733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:24:01.892733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:24:01.892733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:24:01.892733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:24:01.892733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:24:01.892733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:24:01.892733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:24:01.892733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:24:01.892733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:24:01.892733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:24:01.892733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:24:01.892733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:24:01.892733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:24:01.894232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:24:01.894232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:24:01.894232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:24:01.894232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:24:01.894232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:24:01.894232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:24:01.894232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:24:01.894232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:24:01.894232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:24:01.894232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:24:01.894232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:24:01.894232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:24:01.894232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 16:24:01.894232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:24:01.894232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 532/1059] 10.222.0.74 () {56 vars in 1105 bytes} [Wed Jul 27 16:24:01 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:24:01.897272 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2640bee8-be37-4ca9-831a-6077931aa3ea None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:01.898809 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508798eb-cb27-44be-a2b5-ec0ab5bcf05d None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:01.933317 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188f942-454b-438e-8ee5-a08f0e900c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:01.933931 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188f942-454b-438e-8ee5-a08f0e900c4e None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:01.934209 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188f942-454b-438e-8ee5-a08f0e900c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:01.934483 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188f942-454b-438e-8ee5-a08f0e900c4e None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:01.934870 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:01.935217 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188f942-454b-438e-8ee5-a08f0e900c4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:01.935518 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188f942-454b-438e-8ee5-a08f0e900c4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:01.937320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188f942-454b-438e-8ee5-a08f0e900c4e None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:01.949995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 528/1060] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:24:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:01.956172 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188f942-454b-438e-8ee5-a08f0e900c4e None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:02.012713 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508798eb-cb27-44be-a2b5-ec0ab5bcf05d None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:02.032500 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:02.049517 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c80b7-3e48-47af-bd22-df892916f338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:02.050223 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c80b7-3e48-47af-bd22-df892916f338 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:02.050583 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c80b7-3e48-47af-bd22-df892916f338 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:02.050948 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c80b7-3e48-47af-bd22-df892916f338 None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:02.051677 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c80b7-3e48-47af-bd22-df892916f338 None admin] RBAC: Adding query filter params (name=image) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:02.052055 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c80b7-3e48-47af-bd22-df892916f338 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:02.053574 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c80b7-3e48-47af-bd22-df892916f338 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:02.059045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 533/1061] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:24:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:24:02.062641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188f942-454b-438e-8ee5-a08f0e900c4e None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:02.064942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c80b7-3e48-47af-bd22-df892916f338 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:02.097913 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666fc42-6f40-40d4-834e-aacdec1d277d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:02.098581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666fc42-6f40-40d4-834e-aacdec1d277d None admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:02.098861 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666fc42-6f40-40d4-834e-aacdec1d277d None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:02.099136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666fc42-6f40-40d4-834e-aacdec1d277d None admin] PATH_INFO: `/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/56dfafd15e284b7986ba07dd3819bf65/roles/2bafbadedabe4d7681043ae2e7cadacd` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:02.101419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666fc42-6f40-40d4-834e-aacdec1d277d None admin] RBAC: Authorizing `identity:create_grant(project_id=8d6bcdd09cf24a5c8003951db37020cf, user_id=56dfafd15e284b7986ba07dd3819bf65, role_id=2bafbadedabe4d7681043ae2e7cadacd)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:02.102905 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666fc42-6f40-40d4-834e-aacdec1d277d None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:02.117067 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 529/1062] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:24:01 2022] PUT /identity/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/56dfafd15e284b7986ba07dd3819bf65/roles/2bafbadedabe4d7681043ae2e7cadacd => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:24:02.186990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:02.191515 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c80b7-3e48-47af-bd22-df892916f338 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:02.230487 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1da9f7-8791-4514-bdb5-5bca0e9e87f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:02.231126 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1da9f7-8791-4514-bdb5-5bca0e9e87f5 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:02.231474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1da9f7-8791-4514-bdb5-5bca0e9e87f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:02.231948 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1da9f7-8791-4514-bdb5-5bca0e9e87f5 None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:02.232718 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1da9f7-8791-4514-bdb5-5bca0e9e87f5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:02.233130 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1da9f7-8791-4514-bdb5-5bca0e9e87f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:02.234607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1da9f7-8791-4514-bdb5-5bca0e9e87f5 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:02.245296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 534/1063] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:24:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:02.247216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666fc42-6f40-40d4-834e-aacdec1d277d None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:02.353161 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666fc42-6f40-40d4-834e-aacdec1d277d None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:02.388864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b727f90-9a65-4d0b-9da6-085c37fa0396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:02.389517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b727f90-9a65-4d0b-9da6-085c37fa0396 None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:02.389801 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b727f90-9a65-4d0b-9da6-085c37fa0396 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:02.390802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b727f90-9a65-4d0b-9da6-085c37fa0396 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:02.391534 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b727f90-9a65-4d0b-9da6-085c37fa0396 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:02.392927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b727f90-9a65-4d0b-9da6-085c37fa0396 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:02.399066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:02.424920 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 530/1064] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:24:02 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:24:02.738754 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440df37-bf10-4489-8d5b-7343f3930029 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:02.739308 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440df37-bf10-4489-8d5b-7343f3930029 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:02.739308 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440df37-bf10-4489-8d5b-7343f3930029 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:02.741242 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 535/1065] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:24:02.747104 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9648a-d89b-4a0e-9df2-e921458dfccc None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:02.747403 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9648a-d89b-4a0e-9df2-e921458dfccc None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:02.747686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9648a-d89b-4a0e-9df2-e921458dfccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:02.772964 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-b7d9648a-d89b-4a0e-9df2-e921458dfccc None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:02.774727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b7d9648a-d89b-4a0e-9df2-e921458dfccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:02.862754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 531/1066] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:24:02.886661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440df37-bf10-4489-8d5b-7343f3930029 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:02.987487 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440df37-bf10-4489-8d5b-7343f3930029 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:03.019941 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32afa272-5fef-40dd-ac79-922dcdc59995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:03.020512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32afa272-5fef-40dd-ac79-922dcdc59995 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:03.020795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32afa272-5fef-40dd-ac79-922dcdc59995 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:03.021092 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32afa272-5fef-40dd-ac79-922dcdc59995 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:03.031927 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32afa272-5fef-40dd-ac79-922dcdc59995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:24:03.036807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-32afa272-5fef-40dd-ac79-922dcdc59995 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:24:03.036807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:03.036807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:24:03.036807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:24:03.036807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:24:03.036807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:24:03.036807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:24:03.036807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:24:03.036807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:24:03.036807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:24:03.036807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:24:03.036807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:24:03.036807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:24:03.036807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:24:03.036807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:24:03.036807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:24:03.037751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:24:03.037751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:24:03.037751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:24:03.037751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:24:03.037751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:24:03.037751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:24:03.037751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:24:03.037751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:24:03.037751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:24:03.037751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:24:03.037751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:24:03.037751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:24:03.037751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:24:03.037751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:24:03.037751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:24:03.037751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:24:03.038602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:24:03.038602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:24:03.038602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:24:03.038602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:24:03.038602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:24:03.038602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:24:03.038602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:24:03.038602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:24:03.038602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:24:03.038602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:24:03.038602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:24:03.038602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:24:03.039990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 536/1067] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:24:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:24:03.044038 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9648a-d89b-4a0e-9df2-e921458dfccc None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:03.167556 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9648a-d89b-4a0e-9df2-e921458dfccc None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:03.191796 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:03.218178 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519e50c-845a-4ef1-9fff-ab773bf47a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:03.218897 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519e50c-845a-4ef1-9fff-ab773bf47a5a None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:03.219252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519e50c-845a-4ef1-9fff-ab773bf47a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:03.219600 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519e50c-845a-4ef1-9fff-ab773bf47a5a None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:03.220627 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519e50c-845a-4ef1-9fff-ab773bf47a5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:03.221212 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519e50c-845a-4ef1-9fff-ab773bf47a5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:03.223431 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519e50c-845a-4ef1-9fff-ab773bf47a5a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:03.235126 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 532/1068] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:24:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:03.242793 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32afa272-5fef-40dd-ac79-922dcdc59995 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:03.268347 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:03.345551 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32afa272-5fef-40dd-ac79-922dcdc59995 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:03.378571 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81777a70-9915-4798-a6a2-cd841a61fc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:03.379211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81777a70-9915-4798-a6a2-cd841a61fc8a None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:03.379491 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81777a70-9915-4798-a6a2-cd841a61fc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:03.379768 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81777a70-9915-4798-a6a2-cd841a61fc8a None admin] PATH_INFO: `/v3/users/84c28997f8654d43b130260b3b62f251` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:03.392999 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81777a70-9915-4798-a6a2-cd841a61fc8a None admin] RBAC: Authorizing `identity:get_user(user_id=84c28997f8654d43b130260b3b62f251)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:03.394448 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81777a70-9915-4798-a6a2-cd841a61fc8a None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:03.396285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 537/1069] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:24:03 2022] GET /identity/v3/users/84c28997f8654d43b130260b3b62f251 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:03.401862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519e50c-845a-4ef1-9fff-ab773bf47a5a None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:03.501820 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519e50c-845a-4ef1-9fff-ab773bf47a5a None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:03.536640 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c83b27-c819-4b40-89c7-d3740d47c3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:03.537316 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c83b27-c819-4b40-89c7-d3740d47c3ac None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:03.537598 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c83b27-c819-4b40-89c7-d3740d47c3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:03.537869 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c83b27-c819-4b40-89c7-d3740d47c3ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:03.593181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:03.616343 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c83b27-c819-4b40-89c7-d3740d47c3ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:03.617778 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c83b27-c819-4b40-89c7-d3740d47c3ac None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:03.720805 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 533/1070] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:24:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 16:24:03.722489 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81777a70-9915-4798-a6a2-cd841a61fc8a None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:03.805532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:03.828744 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81777a70-9915-4798-a6a2-cd841a61fc8a None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:03.863826 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8dc3fa-d1e7-49ac-8ff1-60b2ac4d194f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:03.864466 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dc3fa-d1e7-49ac-8ff1-60b2ac4d194f None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:03.864743 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dc3fa-d1e7-49ac-8ff1-60b2ac4d194f None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:03.865051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dc3fa-d1e7-49ac-8ff1-60b2ac4d194f None admin] PATH_INFO: `/v3/projects/6061e445a046473ca280fc4eec54c559` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:03.866203 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8dc3fa-d1e7-49ac-8ff1-60b2ac4d194f None admin] RBAC: Authorizing `identity:get_project(project_id=6061e445a046473ca280fc4eec54c559)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:03.867689 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8dc3fa-d1e7-49ac-8ff1-60b2ac4d194f None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:03.870041 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 538/1071] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:24:03 2022] GET /identity/v3/projects/6061e445a046473ca280fc4eec54c559 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:03.876044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c83b27-c819-4b40-89c7-d3740d47c3ac None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:03.955094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:03.978444 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c83b27-c819-4b40-89c7-d3740d47c3ac None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:04.014573 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b5cb1-2b09-4165-bffd-bfc26b44c41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:04.015471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b5cb1-2b09-4165-bffd-bfc26b44c41e None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:04.016015 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b5cb1-2b09-4165-bffd-bfc26b44c41e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:04.016469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b5cb1-2b09-4165-bffd-bfc26b44c41e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:04.017445 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b5cb1-2b09-4165-bffd-bfc26b44c41e None admin] RBAC: Adding query filter params (role.id=ca37e43082fa40c6a88b12680a0ca94b, scope.project.id=6061e445a046473ca280fc4eec54c559, user.id=84c28997f8654d43b130260b3b62f251) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:04.017910 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b5cb1-2b09-4165-bffd-bfc26b44c41e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:04.019491 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b5cb1-2b09-4165-bffd-bfc26b44c41e None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:04.025619 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 534/1072] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:24:03 2022] GET /identity/v3/role_assignments?user.id=84c28997f8654d43b130260b3b62f251&scope.project.id=6061e445a046473ca280fc4eec54c559&role.id=ca37e43082fa40c6a88b12680a0ca94b => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:04.138285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ebf06-fb16-4044-ae0e-f4250e5412b8 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:04.138465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ebf06-fb16-4044-ae0e-f4250e5412b8 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:04.138747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ebf06-fb16-4044-ae0e-f4250e5412b8 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:04.140321 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 539/1073] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:24:04.145665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2268e6b-f14d-4899-a89e-265f698b613e None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:04.145948 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2268e6b-f14d-4899-a89e-265f698b613e None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:04.146226 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2268e6b-f14d-4899-a89e-265f698b613e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:04.172314 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-b2268e6b-f14d-4899-a89e-265f698b613e None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:04.173262 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b2268e6b-f14d-4899-a89e-265f698b613e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:04.242755 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 535/1074] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:24:04.267751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ebf06-fb16-4044-ae0e-f4250e5412b8 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:04.376484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ebf06-fb16-4044-ae0e-f4250e5412b8 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:04.410794 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5b59f-a949-4141-b223-6c6295e9857f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:04.411367 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5b59f-a949-4141-b223-6c6295e9857f None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:04.411650 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5b59f-a949-4141-b223-6c6295e9857f None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:04.411926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5b59f-a949-4141-b223-6c6295e9857f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:04.422758 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee5b59f-a949-4141-b223-6c6295e9857f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:24:04.427747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-1ee5b59f-a949-4141-b223-6c6295e9857f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 16:24:04.427747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:04.427747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:24:04.427747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:24:04.427747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:24:04.427747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:24:04.427747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:24:04.427747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:24:04.427747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:24:04.427747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:24:04.427747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:24:04.427747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:24:04.427747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:24:04.427747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:24:04.427747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:24:04.427747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:24:04.428659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:24:04.428659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:24:04.428659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:24:04.428659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:24:04.428659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:24:04.428659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:24:04.428659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:24:04.428659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:24:04.428659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:24:04.428659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:24:04.428659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:24:04.428659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:24:04.428659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:24:04.428659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:24:04.428659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:24:04.428659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:24:04.435119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:24:04.435119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:24:04.435119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:24:04.435119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:24:04.435119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:24:04.435119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:24:04.435119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:24:04.435119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:24:04.435119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:24:04.435119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:24:04.435119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 16:24:04.435119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:24:04.435119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 540/1075] 10.222.0.74 () {56 vars in 1120 bytes} [Wed Jul 27 16:24:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:24:04.437355 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2268e6b-f14d-4899-a89e-265f698b613e None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:04.512832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:04.531904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd858e-9c7e-4390-8854-fe72bdf2473f None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:04.532370 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd858e-9c7e-4390-8854-fe72bdf2473f None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:04.532657 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd858e-9c7e-4390-8854-fe72bdf2473f None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:04.534173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 541/1076] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:24:04.538818 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01690f23-9d67-4c6c-bebf-b0c96d091628 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:04.539073 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01690f23-9d67-4c6c-bebf-b0c96d091628 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:04.539360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01690f23-9d67-4c6c-bebf-b0c96d091628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:04.545246 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2268e6b-f14d-4899-a89e-265f698b613e None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:04.557971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-01690f23-9d67-4c6c-bebf-b0c96d091628 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:04.559165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-01690f23-9d67-4c6c-bebf-b0c96d091628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:04.561214 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:04.583639 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b3011-5f44-4a1a-b461-54588f895f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:04.584266 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b3011-5f44-4a1a-b461-54588f895f7d None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:04.584547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b3011-5f44-4a1a-b461-54588f895f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:04.584821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b3011-5f44-4a1a-b461-54588f895f7d None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:04.585561 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b3011-5f44-4a1a-b461-54588f895f7d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:04.585869 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b3011-5f44-4a1a-b461-54588f895f7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:04.587233 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b3011-5f44-4a1a-b461-54588f895f7d None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:04.595911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 536/1077] 10.222.0.74 () {56 vars in 1115 bytes} [Wed Jul 27 16:24:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:24:04.601574 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b3011-5f44-4a1a-b461-54588f895f7d None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:04.643717 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 542/1078] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 16:24:04.667100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01690f23-9d67-4c6c-bebf-b0c96d091628 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:04.708579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b3011-5f44-4a1a-b461-54588f895f7d None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:04.745612 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4be6d9-7cb7-426d-b9be-2bc269a0848f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:04.746259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4be6d9-7cb7-426d-b9be-2bc269a0848f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:04.746540 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4be6d9-7cb7-426d-b9be-2bc269a0848f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:04.746811 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4be6d9-7cb7-426d-b9be-2bc269a0848f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:04.748839 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:04.758561 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4be6d9-7cb7-426d-b9be-2bc269a0848f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:04.760059 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4be6d9-7cb7-426d-b9be-2bc269a0848f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:04.774223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-4c4be6d9-7cb7-426d-b9be-2bc269a0848f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:24:04.774223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:04.774223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:24:04.774223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:24:04.774223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:24:04.774223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:24:04.774223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:24:04.774223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:24:04.774223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:24:04.774223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:24:04.774223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:24:04.774223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:24:04.774223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:24:04.774223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:24:04.774223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:24:04.775115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:24:04.775115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:24:04.775115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:24:04.775115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:24:04.775115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:24:04.775115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:24:04.775115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:24:04.775115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:24:04.775115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:24:04.775115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:24:04.775115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:24:04.775115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:24:04.775115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:24:04.775115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:24:04.775115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:24:04.775115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:24:04.775115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:24:04.776629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:24:04.776629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:24:04.776629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:24:04.776629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:24:04.776629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 537/1079] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:24:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:24:04.780849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01690f23-9d67-4c6c-bebf-b0c96d091628 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:04.781471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4be6d9-7cb7-426d-b9be-2bc269a0848f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:04.822245 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d0515-4cd9-4c81-833d-962bd088f8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:04.822866 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d0515-4cd9-4c81-833d-962bd088f8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:04.823146 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d0515-4cd9-4c81-833d-962bd088f8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:04.823416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d0515-4cd9-4c81-833d-962bd088f8c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:04.832366 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:04.890589 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4be6d9-7cb7-426d-b9be-2bc269a0848f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:04.893867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d0515-4cd9-4c81-833d-962bd088f8c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:04.895318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d0515-4cd9-4c81-833d-962bd088f8c0 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:04.928079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a753c-6a90-49ed-b234-c1d6a57bb340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:04.928632 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a753c-6a90-49ed-b234-c1d6a57bb340 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:04.928937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a753c-6a90-49ed-b234-c1d6a57bb340 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:04.929218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a753c-6a90-49ed-b234-c1d6a57bb340 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:04.929909 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a753c-6a90-49ed-b234-c1d6a57bb340 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:04.930213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a753c-6a90-49ed-b234-c1d6a57bb340 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:04.931615 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a753c-6a90-49ed-b234-c1d6a57bb340 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:04.944264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 538/1080] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:24:04.951095 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a753c-6a90-49ed-b234-c1d6a57bb340 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:04.999582 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:05.010026 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 543/1081] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:24:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 16:24:05.011947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d0515-4cd9-4c81-833d-962bd088f8c0 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:05.052325 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:05.057581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a753c-6a90-49ed-b234-c1d6a57bb340 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:05.095680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19d7ff-625d-4475-b264-6b6da648e58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:05.096326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19d7ff-625d-4475-b264-6b6da648e58d None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:05.096614 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19d7ff-625d-4475-b264-6b6da648e58d None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:05.096922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19d7ff-625d-4475-b264-6b6da648e58d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:05.104373 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e19d7ff-625d-4475-b264-6b6da648e58d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:05.105909 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e19d7ff-625d-4475-b264-6b6da648e58d None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:05.119495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-8e19d7ff-625d-4475-b264-6b6da648e58d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 16:24:05.119495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:05.119495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:24:05.119495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:24:05.119495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:24:05.119495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:24:05.119495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:24:05.119495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:24:05.119495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:24:05.119495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:24:05.119495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:24:05.119495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:24:05.119495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:24:05.119495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:24:05.119495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:24:05.119495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:24:05.120507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:24:05.120507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:24:05.120507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:24:05.120507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:24:05.120507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:24:05.120507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:24:05.120507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:24:05.120507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:24:05.120507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:24:05.120507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:24:05.120507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:24:05.120507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:24:05.120507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:24:05.120507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:24:05.120507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:24:05.120507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:24:05.120507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:24:05.120507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:24:05.121818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:24:05.121818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:24:05.121818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:24:05.121818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:24:05.121818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:24:05.121818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:24:05.121818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:24:05.121818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:24:05.121818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 16:24:05.121818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:24:05.121818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 539/1082] 10.222.0.74 () {56 vars in 1093 bytes} [Wed Jul 27 16:24:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:24:05.125416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d0515-4cd9-4c81-833d-962bd088f8c0 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:05.126318 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19d7ff-625d-4475-b264-6b6da648e58d None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:05.163108 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4344a-3fff-419f-aa10-a7f50ef59666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:05.164060 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4344a-3fff-419f-aa10-a7f50ef59666 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:05.164337 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4344a-3fff-419f-aa10-a7f50ef59666 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:05.164610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4344a-3fff-419f-aa10-a7f50ef59666 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:05.174740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4344a-3fff-419f-aa10-a7f50ef59666 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:05.176193 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4344a-3fff-419f-aa10-a7f50ef59666 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:05.190940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-b0e4344a-3fff-419f-aa10-a7f50ef59666 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:24:05.190940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:05.190940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:24:05.190940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:24:05.190940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:24:05.190940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:24:05.190940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:24:05.190940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:24:05.190940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:24:05.190940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:24:05.190940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:24:05.190940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:24:05.190940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:24:05.190940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:24:05.190940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:24:05.191833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:24:05.191833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:24:05.191833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:24:05.191833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:24:05.191833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:24:05.191833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:24:05.191833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:24:05.191833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:24:05.191833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:24:05.191833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:24:05.191833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:24:05.191833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:24:05.191833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:24:05.191833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:24:05.191833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:24:05.191833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:24:05.191833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:24:05.192738 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:24:05.192738 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:24:05.192738 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:24:05.192738 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:24:05.196796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 544/1083] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:24:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:24:05.198605 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4344a-3fff-419f-aa10-a7f50ef59666 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:05.245554 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19d7ff-625d-4475-b264-6b6da648e58d None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:05.253002 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:05.283360 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa64ebc-4df2-4c57-9796-21cff7563dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:05.283973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa64ebc-4df2-4c57-9796-21cff7563dcf None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:05.284251 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa64ebc-4df2-4c57-9796-21cff7563dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:05.284523 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa64ebc-4df2-4c57-9796-21cff7563dcf None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:05.285333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa64ebc-4df2-4c57-9796-21cff7563dcf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:05.285635 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa64ebc-4df2-4c57-9796-21cff7563dcf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:05.287070 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa64ebc-4df2-4c57-9796-21cff7563dcf None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:05.304390 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:05.307382 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 540/1084] 10.222.0.74 () {56 vars in 1133 bytes} [Wed Jul 27 16:24:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:24:05.312736 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa64ebc-4df2-4c57-9796-21cff7563dcf None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:05.315243 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4344a-3fff-419f-aa10-a7f50ef59666 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:05.352335 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1859c6bc-b32e-41db-afda-8db9aee28e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:05.352975 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859c6bc-b32e-41db-afda-8db9aee28e5f None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:05.353257 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859c6bc-b32e-41db-afda-8db9aee28e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:05.353554 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859c6bc-b32e-41db-afda-8db9aee28e5f None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:05.354237 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1859c6bc-b32e-41db-afda-8db9aee28e5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:05.354539 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1859c6bc-b32e-41db-afda-8db9aee28e5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:05.355897 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1859c6bc-b32e-41db-afda-8db9aee28e5f None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:05.367728 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 545/1085] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:05.420242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa64ebc-4df2-4c57-9796-21cff7563dcf None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:05.459252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0dcce-a4d4-4990-97e4-1e581b97fafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:05.459923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0dcce-a4d4-4990-97e4-1e581b97fafe None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:05.460145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0dcce-a4d4-4990-97e4-1e581b97fafe None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:05.460420 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0dcce-a4d4-4990-97e4-1e581b97fafe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:05.493441 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:05.537035 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0dcce-a4d4-4990-97e4-1e581b97fafe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:05.538497 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0dcce-a4d4-4990-97e4-1e581b97fafe None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:05.642850 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 541/1086] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:24:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 16:24:05.648671 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1859c6bc-b32e-41db-afda-8db9aee28e5f None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:05.751052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1859c6bc-b32e-41db-afda-8db9aee28e5f None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:05.783244 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624bd90-427e-4b68-88e1-a80c37c63648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:05.783872 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624bd90-427e-4b68-88e1-a80c37c63648 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:05.784149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624bd90-427e-4b68-88e1-a80c37c63648 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:05.784420 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624bd90-427e-4b68-88e1-a80c37c63648 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:05.794664 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624bd90-427e-4b68-88e1-a80c37c63648 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:05.796097 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624bd90-427e-4b68-88e1-a80c37c63648 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:05.811012 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-0624bd90-427e-4b68-88e1-a80c37c63648 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:24:05.811012 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:05.811012 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:24:05.811012 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:24:05.811012 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:24:05.811012 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:24:05.811012 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:24:05.811012 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:24:05.811012 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:24:05.811012 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:24:05.811012 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:24:05.811012 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:24:05.811012 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:24:05.811012 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:24:05.811012 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:24:05.812045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:24:05.812045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:24:05.812045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:24:05.812045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:24:05.812045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:24:05.812045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:24:05.812045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:24:05.812045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:24:05.812045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:24:05.812045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:24:05.812045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:24:05.812045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:24:05.812045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:24:05.812045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:24:05.812045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:24:05.812045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:24:05.812045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:24:05.812963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:24:05.812963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:24:05.812963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:24:05.812963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:24:05.812963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 546/1087] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:24:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:24:05.817823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0dcce-a4d4-4990-97e4-1e581b97fafe None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:05.877470 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:05.914338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0dcce-a4d4-4990-97e4-1e581b97fafe None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:05.948942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c5119-98c7-4b3a-b6ce-953ba7e013c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:05.949566 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c5119-98c7-4b3a-b6ce-953ba7e013c4 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:05.949842 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c5119-98c7-4b3a-b6ce-953ba7e013c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:05.950117 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c5119-98c7-4b3a-b6ce-953ba7e013c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:05.951062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c5119-98c7-4b3a-b6ce-953ba7e013c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:05.951646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c5119-98c7-4b3a-b6ce-953ba7e013c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:05.953129 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c5119-98c7-4b3a-b6ce-953ba7e013c4 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:05.964668 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 542/1088] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:05.970327 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624bd90-427e-4b68-88e1-a80c37c63648 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:05.973260 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:06.022649 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a52a3-ac98-43b2-b2b6-1828de36cfa9 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:06.022844 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a52a3-ac98-43b2-b2b6-1828de36cfa9 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:06.023118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a52a3-ac98-43b2-b2b6-1828de36cfa9 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:06.024461 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 543/1089] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:24:06.029838 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1d9aa-f2fa-49b3-aac0-036628d07743 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:06.030120 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1d9aa-f2fa-49b3-aac0-036628d07743 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:06.030400 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1d9aa-f2fa-49b3-aac0-036628d07743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:06.048836 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-88d1d9aa-f2fa-49b3-aac0-036628d07743 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:06.050055 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-88d1d9aa-f2fa-49b3-aac0-036628d07743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:06.072292 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624bd90-427e-4b68-88e1-a80c37c63648 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:06.106383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f04e16-9ace-4e23-9564-e2534881ef4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:06.107025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f04e16-9ace-4e23-9564-e2534881ef4b None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:06.107309 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f04e16-9ace-4e23-9564-e2534881ef4b None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:06.107586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f04e16-9ace-4e23-9564-e2534881ef4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:06.114245 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f04e16-9ace-4e23-9564-e2534881ef4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:06.115690 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f04e16-9ace-4e23-9564-e2534881ef4b None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:06.115833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 544/1090] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 16:24:06.127901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-81f04e16-9ace-4e23-9564-e2534881ef4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:24:06.127901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:06.127901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:24:06.127901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:24:06.127901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:24:06.127901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:24:06.127901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:24:06.127901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:24:06.127901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:24:06.127901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:24:06.127901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:24:06.127901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:24:06.127901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:24:06.127901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:24:06.127901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:24:06.129120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:24:06.129120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:24:06.129120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:24:06.129120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:24:06.129120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:24:06.129120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:24:06.129120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:24:06.129120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:24:06.129120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:24:06.129120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:24:06.129120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:24:06.129120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:24:06.129120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:24:06.129120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:24:06.129120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:24:06.129120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:24:06.129120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:24:06.130225 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:24:06.130225 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:24:06.130225 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:24:06.130225 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:24:06.130225 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:24:06.130225 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:24:06.130225 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:24:06.130225 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:24:06.130225 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 547/1091] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:24:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:24:06.134700 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d1d9aa-f2fa-49b3-aac0-036628d07743 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:06.137875 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f04e16-9ace-4e23-9564-e2534881ef4b None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:06.192586 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:06.200560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:06.237919 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d1d9aa-f2fa-49b3-aac0-036628d07743 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:06.239991 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f04e16-9ace-4e23-9564-e2534881ef4b None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:06.272119 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d053726-4ac0-4c35-b491-adb802505a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:06.272782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d053726-4ac0-4c35-b491-adb802505a20 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:06.272926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eac566-3911-4371-8b52-628444acab6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:06.273056 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d053726-4ac0-4c35-b491-adb802505a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:06.273299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d053726-4ac0-4c35-b491-adb802505a20 None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:06.273560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eac566-3911-4371-8b52-628444acab6a None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:06.273842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eac566-3911-4371-8b52-628444acab6a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:06.274036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d053726-4ac0-4c35-b491-adb802505a20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:06.274129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eac566-3911-4371-8b52-628444acab6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:06.274338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d053726-4ac0-4c35-b491-adb802505a20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:06.276334 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d053726-4ac0-4c35-b491-adb802505a20 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:06.284445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eac566-3911-4371-8b52-628444acab6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:24:06.287416 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 545/1092] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:24:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:24:06.291747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-a6eac566-3911-4371-8b52-628444acab6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:24:06.291747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:06.291747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:24:06.291747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:24:06.291747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:24:06.291747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:24:06.291747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:24:06.291747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:24:06.291747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:24:06.291747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:24:06.291747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:24:06.291747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:24:06.291747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:24:06.291747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:24:06.291747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:24:06.291747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:24:06.292773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:24:06.292773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:24:06.292773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:24:06.292773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:24:06.292773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:24:06.292773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:24:06.292773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:24:06.292773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:24:06.292773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:24:06.292773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:24:06.292773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:24:06.292773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:24:06.292773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:24:06.292773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:24:06.292773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:24:06.292773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:24:06.293681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:24:06.293681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:24:06.293681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:24:06.293681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:24:06.293681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:24:06.293681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:24:06.293681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:24:06.293681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:24:06.293681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:24:06.293681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:24:06.293681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:24:06.293681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:24:06.293681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 548/1093] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:24:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:24:06.294400 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d053726-4ac0-4c35-b491-adb802505a20 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:06.297368 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eac566-3911-4371-8b52-628444acab6a None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:06.392247 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d053726-4ac0-4c35-b491-adb802505a20 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:06.395227 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eac566-3911-4371-8b52-628444acab6a None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:06.405918 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:06.417865 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:06.426468 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e36d8-6d4f-434e-8fa9-5bc883fddc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:06.427094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e36d8-6d4f-434e-8fa9-5bc883fddc1d None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:06.427376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e36d8-6d4f-434e-8fa9-5bc883fddc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:06.427651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e36d8-6d4f-434e-8fa9-5bc883fddc1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:06.428484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e36d8-6d4f-434e-8fa9-5bc883fddc1d None admin] RBAC: Adding query filter params (role.id=2bafbadedabe4d7681043ae2e7cadacd, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=56dfafd15e284b7986ba07dd3819bf65) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:06.428742 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e36d8-6d4f-434e-8fa9-5bc883fddc1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:06.430151 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2bba83-941b-4ce2-b137-624bb13aa80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:06.430269 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e36d8-6d4f-434e-8fa9-5bc883fddc1d None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:06.430757 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2bba83-941b-4ce2-b137-624bb13aa80a None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:06.431034 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2bba83-941b-4ce2-b137-624bb13aa80a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:06.431304 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2bba83-941b-4ce2-b137-624bb13aa80a None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:06.432004 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2bba83-941b-4ce2-b137-624bb13aa80a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:06.432303 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2bba83-941b-4ce2-b137-624bb13aa80a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:06.433668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2bba83-941b-4ce2-b137-624bb13aa80a None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:06.435959 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 546/1094] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:24:06 2022] GET /identity/v3/role_assignments?user.id=56dfafd15e284b7986ba07dd3819bf65&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=2bafbadedabe4d7681043ae2e7cadacd => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:24:06.445369 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 549/1095] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:24:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:24:06.452201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e36d8-6d4f-434e-8fa9-5bc883fddc1d None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:06.557742 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e36d8-6d4f-434e-8fa9-5bc883fddc1d None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:06.593273 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3e8a9-5efd-4a25-a667-efad8b7123a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:06.593929 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3e8a9-5efd-4a25-a667-efad8b7123a8 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:06.594214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3e8a9-5efd-4a25-a667-efad8b7123a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:06.594492 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3e8a9-5efd-4a25-a667-efad8b7123a8 None admin] PATH_INFO: `/v3/users/84c28997f8654d43b130260b3b62f251` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:06.595679 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3e8a9-5efd-4a25-a667-efad8b7123a8 None admin] RBAC: Authorizing `identity:get_user(user_id=84c28997f8654d43b130260b3b62f251)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:06.597169 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3e8a9-5efd-4a25-a667-efad8b7123a8 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:06.598809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 547/1096] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:24:06 2022] GET /identity/v3/users/84c28997f8654d43b130260b3b62f251 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:06.604883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2bba83-941b-4ce2-b137-624bb13aa80a None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:06.708381 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2bba83-941b-4ce2-b137-624bb13aa80a None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:06.742571 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324ed1e-37b3-4814-924d-5fde5f47ec0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:06.743217 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324ed1e-37b3-4814-924d-5fde5f47ec0c None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:06.743491 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324ed1e-37b3-4814-924d-5fde5f47ec0c None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:06.743763 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324ed1e-37b3-4814-924d-5fde5f47ec0c None admin] PATH_INFO: `/v3/projects/6061e445a046473ca280fc4eec54c559` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:06.745511 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324ed1e-37b3-4814-924d-5fde5f47ec0c None admin] RBAC: Authorizing `identity:get_project(project_id=6061e445a046473ca280fc4eec54c559)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:06.746936 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324ed1e-37b3-4814-924d-5fde5f47ec0c None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:06.748590 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 550/1097] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:24:06 2022] GET /identity/v3/projects/6061e445a046473ca280fc4eec54c559 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:06.754737 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3e8a9-5efd-4a25-a667-efad8b7123a8 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:06.802968 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:06.860077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3e8a9-5efd-4a25-a667-efad8b7123a8 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:06.895297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c9460-df81-481d-b75b-a96f324d094a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:06.895976 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c9460-df81-481d-b75b-a96f324d094a None admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:06.896261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c9460-df81-481d-b75b-a96f324d094a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:06.896542 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c9460-df81-481d-b75b-a96f324d094a None admin] PATH_INFO: `/v3/projects/6061e445a046473ca280fc4eec54c559/users/84c28997f8654d43b130260b3b62f251/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:06.898522 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c9460-df81-481d-b75b-a96f324d094a None admin] RBAC: Authorizing `identity:create_grant(project_id=6061e445a046473ca280fc4eec54c559, user_id=84c28997f8654d43b130260b3b62f251, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:06.899959 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c9460-df81-481d-b75b-a96f324d094a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:06.912996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 548/1098] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:24:06 2022] PUT /identity/v3/projects/6061e445a046473ca280fc4eec54c559/users/84c28997f8654d43b130260b3b62f251/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:24:08.857727 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9de14a-b13f-4a9a-82d9-d8d22315c372 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:08.858254 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9de14a-b13f-4a9a-82d9-d8d22315c372 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:08.858254 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9de14a-b13f-4a9a-82d9-d8d22315c372 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:08.859599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 551/1099] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:24:08.864713 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa2e24-5d8b-4061-a57c-52ac73e5c699 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:08.865147 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa2e24-5d8b-4061-a57c-52ac73e5c699 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:08.865318 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa2e24-5d8b-4061-a57c-52ac73e5c699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:08.883322 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-d7fa2e24-5d8b-4061-a57c-52ac73e5c699 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:08.885036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d7fa2e24-5d8b-4061-a57c-52ac73e5c699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:08.948579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 549/1100] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:24:08.972055 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9de14a-b13f-4a9a-82d9-d8d22315c372 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:09.030015 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:09.076939 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9de14a-b13f-4a9a-82d9-d8d22315c372 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:09.109387 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e273af-651b-4eaf-979f-c3183d28fa3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:09.109948 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e273af-651b-4eaf-979f-c3183d28fa3b None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:09.110225 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e273af-651b-4eaf-979f-c3183d28fa3b None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:09.110497 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e273af-651b-4eaf-979f-c3183d28fa3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:09.120630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e273af-651b-4eaf-979f-c3183d28fa3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:24:09.125494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-a6e273af-651b-4eaf-979f-c3183d28fa3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:24:09.125494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:09.125494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:24:09.125494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:24:09.125494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:24:09.125494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:24:09.125494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:24:09.125494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:24:09.125494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:24:09.125494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:24:09.125494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:24:09.125494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:24:09.125494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:24:09.125494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:24:09.125494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:24:09.125494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:24:09.126564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:24:09.126564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:24:09.126564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:24:09.126564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:24:09.126564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:24:09.126564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:24:09.126564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:24:09.126564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:24:09.126564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:24:09.126564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:24:09.126564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:24:09.126564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:24:09.126564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:24:09.126564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:24:09.126564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:24:09.126564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:24:09.127589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:24:09.127589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:24:09.127589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:24:09.127589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:24:09.127589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:24:09.127589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:24:09.127589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:24:09.127589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:24:09.127589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:24:09.127589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:24:09.127589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:24:09.127589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:24:09.127589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 552/1101] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:24:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:24:09.132611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa2e24-5d8b-4061-a57c-52ac73e5c699 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:09.166808 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:09.230746 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa2e24-5d8b-4061-a57c-52ac73e5c699 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:09.266170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07d6d6-b07c-462e-aa49-484368543cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:09.266776 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07d6d6-b07c-462e-aa49-484368543cde None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:09.267051 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07d6d6-b07c-462e-aa49-484368543cde None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:09.267319 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07d6d6-b07c-462e-aa49-484368543cde None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:09.268030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07d6d6-b07c-462e-aa49-484368543cde None admin] RBAC: Adding query filter params (name=admin) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:09.268329 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07d6d6-b07c-462e-aa49-484368543cde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:09.269722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07d6d6-b07c-462e-aa49-484368543cde None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:09.277510 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 550/1102] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:24:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:09.284470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e273af-651b-4eaf-979f-c3183d28fa3b None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:09.381926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e273af-651b-4eaf-979f-c3183d28fa3b None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:09.394987 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:09.415248 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62500d1-05d3-4cb1-a86b-4f3d300f809a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:09.416382 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62500d1-05d3-4cb1-a86b-4f3d300f809a None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:09.416657 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62500d1-05d3-4cb1-a86b-4f3d300f809a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:09.416944 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62500d1-05d3-4cb1-a86b-4f3d300f809a None admin] PATH_INFO: `/v3/users/84c28997f8654d43b130260b3b62f251` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:09.418097 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62500d1-05d3-4cb1-a86b-4f3d300f809a None admin] RBAC: Authorizing `identity:get_user(user_id=84c28997f8654d43b130260b3b62f251)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:09.419516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62500d1-05d3-4cb1-a86b-4f3d300f809a None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:09.421134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 553/1103] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:24:09 2022] GET /identity/v3/users/84c28997f8654d43b130260b3b62f251 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:09.426898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07d6d6-b07c-462e-aa49-484368543cde None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:09.519574 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:09.523907 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07d6d6-b07c-462e-aa49-484368543cde None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:09.556154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609759a0-26fb-4e9e-9286-820f754066ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:09.556839 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609759a0-26fb-4e9e-9286-820f754066ec None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:09.557152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609759a0-26fb-4e9e-9286-820f754066ec None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:09.557430 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609759a0-26fb-4e9e-9286-820f754066ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:09.623860 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609759a0-26fb-4e9e-9286-820f754066ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:09.625350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609759a0-26fb-4e9e-9286-820f754066ec None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:09.716699 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:09.721797 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 551/1104] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:24:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 16:24:09.727269 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62500d1-05d3-4cb1-a86b-4f3d300f809a None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:09.825537 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62500d1-05d3-4cb1-a86b-4f3d300f809a None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:09.857826 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164adf69-41fc-4783-9997-9bb09b3d9e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:09.858432 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164adf69-41fc-4783-9997-9bb09b3d9e7d None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:09.858711 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164adf69-41fc-4783-9997-9bb09b3d9e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:09.858982 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164adf69-41fc-4783-9997-9bb09b3d9e7d None admin] PATH_INFO: `/v3/projects/6061e445a046473ca280fc4eec54c559` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:09.860111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164adf69-41fc-4783-9997-9bb09b3d9e7d None admin] RBAC: Authorizing `identity:get_project(project_id=6061e445a046473ca280fc4eec54c559)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:09.861557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164adf69-41fc-4783-9997-9bb09b3d9e7d None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:09.863195 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 554/1105] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:24:09 2022] GET /identity/v3/projects/6061e445a046473ca280fc4eec54c559 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:09.869839 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609759a0-26fb-4e9e-9286-820f754066ec None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:09.967426 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609759a0-26fb-4e9e-9286-820f754066ec None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:10.000266 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baec73a-9067-4b63-8ad7-64a73412c875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:10.000868 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baec73a-9067-4b63-8ad7-64a73412c875 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:10.001179 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baec73a-9067-4b63-8ad7-64a73412c875 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:10.001476 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baec73a-9067-4b63-8ad7-64a73412c875 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:10.002274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baec73a-9067-4b63-8ad7-64a73412c875 None admin] RBAC: Adding query filter params (role.id=ca37e43082fa40c6a88b12680a0ca94b, scope.project.id=6061e445a046473ca280fc4eec54c559, user.id=84c28997f8654d43b130260b3b62f251) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:10.002607 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baec73a-9067-4b63-8ad7-64a73412c875 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:10.004063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baec73a-9067-4b63-8ad7-64a73412c875 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:10.010409 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 552/1106] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:24:09 2022] GET /identity/v3/role_assignments?user.id=84c28997f8654d43b130260b3b62f251&scope.project.id=6061e445a046473ca280fc4eec54c559&role.id=ca37e43082fa40c6a88b12680a0ca94b => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:12.011068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f17b4-ef13-4320-a8c4-09c67cd738d7 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:12.011605 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f17b4-ef13-4320-a8c4-09c67cd738d7 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:12.011605 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f17b4-ef13-4320-a8c4-09c67cd738d7 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:12.012959 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 555/1107] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:24:12.018009 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11301513-4c2f-4c3e-b2c7-57719d89e0bd None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:12.018379 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11301513-4c2f-4c3e-b2c7-57719d89e0bd None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:12.018561 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11301513-4c2f-4c3e-b2c7-57719d89e0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:12.036692 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-11301513-4c2f-4c3e-b2c7-57719d89e0bd None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:12.037901 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-11301513-4c2f-4c3e-b2c7-57719d89e0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:12.092165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:12.102548 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 553/1108] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:24:12.126936 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f17b4-ef13-4320-a8c4-09c67cd738d7 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:12.177159 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:12.227243 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f17b4-ef13-4320-a8c4-09c67cd738d7 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:12.260030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b79592-fcc6-49ac-b41c-0f3246c9e7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:12.260653 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b79592-fcc6-49ac-b41c-0f3246c9e7c8 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:12.260953 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b79592-fcc6-49ac-b41c-0f3246c9e7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:12.261294 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b79592-fcc6-49ac-b41c-0f3246c9e7c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:12.262360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b79592-fcc6-49ac-b41c-0f3246c9e7c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:12.263806 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b79592-fcc6-49ac-b41c-0f3246c9e7c8 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:12.265451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 556/1109] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:24:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:12.271236 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11301513-4c2f-4c3e-b2c7-57719d89e0bd None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:12.368660 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11301513-4c2f-4c3e-b2c7-57719d89e0bd None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:12.403713 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d5900f-5939-447a-a550-bf08918bc673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:12.404332 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5900f-5939-447a-a550-bf08918bc673 None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:12.404614 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5900f-5939-447a-a550-bf08918bc673 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:12.404904 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5900f-5939-447a-a550-bf08918bc673 None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:12.405626 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d5900f-5939-447a-a550-bf08918bc673 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:12.407082 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d5900f-5939-447a-a550-bf08918bc673 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:12.426677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 554/1110] 10.222.0.74 () {60 vars in 1132 bytes} [Wed Jul 27 16:24:12 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:24:14.427850 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaace200-3a3f-405c-b3a3-6ce65ec950e1 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:14.428352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaace200-3a3f-405c-b3a3-6ce65ec950e1 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:14.428352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaace200-3a3f-405c-b3a3-6ce65ec950e1 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:14.429678 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 557/1111] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:24:14.435294 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc4208-4555-430f-b3bc-4d79ada6a5dc None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:14.435705 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc4208-4555-430f-b3bc-4d79ada6a5dc None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:14.435856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc4208-4555-430f-b3bc-4d79ada6a5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:14.454453 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-61dc4208-4555-430f-b3bc-4d79ada6a5dc None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:14.455637 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-61dc4208-4555-430f-b3bc-4d79ada6a5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:14.497687 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:14.521759 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 555/1112] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 16:24:14.545786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaace200-3a3f-405c-b3a3-6ce65ec950e1 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:14.646740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaace200-3a3f-405c-b3a3-6ce65ec950e1 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:14.662449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:14.680245 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c7076-d439-4ddb-88ea-54ce30f0c350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:14.680820 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c7076-d439-4ddb-88ea-54ce30f0c350 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:14.681134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c7076-d439-4ddb-88ea-54ce30f0c350 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:14.681467 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c7076-d439-4ddb-88ea-54ce30f0c350 None admin] PATH_INFO: `/v3/roles/70f3e09bc103410789de8e15c81d9cd3` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:14.682918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c7076-d439-4ddb-88ea-54ce30f0c350 None admin] RBAC: Authorizing `identity:get_role(role_id=70f3e09bc103410789de8e15c81d9cd3)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:14.684285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c7076-d439-4ddb-88ea-54ce30f0c350 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:14.685644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 558/1113] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:24:14 2022] GET /identity/v3/roles/70f3e09bc103410789de8e15c81d9cd3 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:14.691600 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc4208-4555-430f-b3bc-4d79ada6a5dc None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:14.787718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc4208-4555-430f-b3bc-4d79ada6a5dc None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:14.821442 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457969d0-9fd3-4dd9-ba01-12c3eb029dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:14.822608 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457969d0-9fd3-4dd9-ba01-12c3eb029dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:14.822890 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457969d0-9fd3-4dd9-ba01-12c3eb029dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:14.823167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457969d0-9fd3-4dd9-ba01-12c3eb029dc4 None admin] PATH_INFO: `/v3/users/3a461301da714fa58c4ea1a7f94aa16c` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:14.836149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457969d0-9fd3-4dd9-ba01-12c3eb029dc4 None admin] RBAC: Authorizing `identity:get_user(user_id=3a461301da714fa58c4ea1a7f94aa16c)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:14.837719 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457969d0-9fd3-4dd9-ba01-12c3eb029dc4 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:14.839295 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 556/1114] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:24:14 2022] GET /identity/v3/users/3a461301da714fa58c4ea1a7f94aa16c => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:14.844745 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c7076-d439-4ddb-88ea-54ce30f0c350 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:14.943214 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c7076-d439-4ddb-88ea-54ce30f0c350 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:14.976919 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ddf637-9dcd-43a0-be4e-aeea720e3895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:14.977597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ddf637-9dcd-43a0-be4e-aeea720e3895 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:14.977883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ddf637-9dcd-43a0-be4e-aeea720e3895 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:14.978158 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ddf637-9dcd-43a0-be4e-aeea720e3895 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:15.028606 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:15.044102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ddf637-9dcd-43a0-be4e-aeea720e3895 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:15.045571 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ddf637-9dcd-43a0-be4e-aeea720e3895 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:15.141943 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 559/1115] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:24:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 16:24:15.147859 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457969d0-9fd3-4dd9-ba01-12c3eb029dc4 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:15.184749 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:15.246993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457969d0-9fd3-4dd9-ba01-12c3eb029dc4 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:15.279755 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd7b2a-9716-4c73-8a65-5546b438a8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:15.280389 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd7b2a-9716-4c73-8a65-5546b438a8ea None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:15.280673 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd7b2a-9716-4c73-8a65-5546b438a8ea None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:15.280976 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd7b2a-9716-4c73-8a65-5546b438a8ea None admin] PATH_INFO: `/v3/projects/6061e445a046473ca280fc4eec54c559` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:15.282112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd7b2a-9716-4c73-8a65-5546b438a8ea None admin] RBAC: Authorizing `identity:get_project(project_id=6061e445a046473ca280fc4eec54c559)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:15.283543 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd7b2a-9716-4c73-8a65-5546b438a8ea None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:15.285223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 557/1116] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:24:15 2022] GET /identity/v3/projects/6061e445a046473ca280fc4eec54c559 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:15.291886 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ddf637-9dcd-43a0-be4e-aeea720e3895 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:15.377835 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:15.393969 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ddf637-9dcd-43a0-be4e-aeea720e3895 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:15.426515 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb830cf-6299-4dfb-b28a-df6cb4f9c005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:15.427121 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb830cf-6299-4dfb-b28a-df6cb4f9c005 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:15.427402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb830cf-6299-4dfb-b28a-df6cb4f9c005 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:15.427677 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb830cf-6299-4dfb-b28a-df6cb4f9c005 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:15.428454 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb830cf-6299-4dfb-b28a-df6cb4f9c005 None admin] RBAC: Adding query filter params (role.id=70f3e09bc103410789de8e15c81d9cd3, scope.project.id=6061e445a046473ca280fc4eec54c559, user.id=3a461301da714fa58c4ea1a7f94aa16c) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:15.429013 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb830cf-6299-4dfb-b28a-df6cb4f9c005 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:15.430200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb830cf-6299-4dfb-b28a-df6cb4f9c005 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:15.435985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 560/1117] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:24:15 2022] GET /identity/v3/role_assignments?user.id=3a461301da714fa58c4ea1a7f94aa16c&scope.project.id=6061e445a046473ca280fc4eec54c559&role.id=70f3e09bc103410789de8e15c81d9cd3 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:17.390059 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623849e1-7698-4a5f-bd8d-670f88760492 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:17.390599 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623849e1-7698-4a5f-bd8d-670f88760492 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:17.390599 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623849e1-7698-4a5f-bd8d-670f88760492 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:17.391936 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 558/1118] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:24:17.397074 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cd6e7-ff02-4e74-bf84-e19421a05f35 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:17.397512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cd6e7-ff02-4e74-bf84-e19421a05f35 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:17.397605 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cd6e7-ff02-4e74-bf84-e19421a05f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:17.415515 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-202cd6e7-ff02-4e74-bf84-e19421a05f35 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:17.417409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-202cd6e7-ff02-4e74-bf84-e19421a05f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:17.480791 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 561/1119] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:24:17.502548 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623849e1-7698-4a5f-bd8d-670f88760492 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:17.600174 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623849e1-7698-4a5f-bd8d-670f88760492 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:17.602460 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:17.633693 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd04726-aafa-4bed-9242-2c9b5ebcd3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:17.634898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd04726-aafa-4bed-9242-2c9b5ebcd3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:17.635195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd04726-aafa-4bed-9242-2c9b5ebcd3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:17.635473 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd04726-aafa-4bed-9242-2c9b5ebcd3b0 None admin] PATH_INFO: `/v3/roles/70f3e09bc103410789de8e15c81d9cd3` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:17.637029 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd04726-aafa-4bed-9242-2c9b5ebcd3b0 None admin] RBAC: Authorizing `identity:get_role(role_id=70f3e09bc103410789de8e15c81d9cd3)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:17.638296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd04726-aafa-4bed-9242-2c9b5ebcd3b0 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:17.639602 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 559/1120] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:24:17 2022] GET /identity/v3/roles/70f3e09bc103410789de8e15c81d9cd3 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:17.645315 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cd6e7-ff02-4e74-bf84-e19421a05f35 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:17.720379 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:17.743090 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cd6e7-ff02-4e74-bf84-e19421a05f35 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:17.776802 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5aff6-fe66-4e60-ade7-7311b94b9638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:17.777495 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5aff6-fe66-4e60-ade7-7311b94b9638 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:17.777779 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5aff6-fe66-4e60-ade7-7311b94b9638 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:17.778056 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5aff6-fe66-4e60-ade7-7311b94b9638 None admin] PATH_INFO: `/v3/users/3a461301da714fa58c4ea1a7f94aa16c` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:17.779245 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5aff6-fe66-4e60-ade7-7311b94b9638 None admin] RBAC: Authorizing `identity:get_user(user_id=3a461301da714fa58c4ea1a7f94aa16c)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:17.780823 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5aff6-fe66-4e60-ade7-7311b94b9638 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:17.782537 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 562/1121] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:24:17 2022] GET /identity/v3/users/3a461301da714fa58c4ea1a7f94aa16c => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:17.788118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd04726-aafa-4bed-9242-2c9b5ebcd3b0 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:17.885043 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd04726-aafa-4bed-9242-2c9b5ebcd3b0 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:17.917567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf99646-02c0-4957-9a14-68f393b63312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:17.918195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf99646-02c0-4957-9a14-68f393b63312 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:17.918477 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf99646-02c0-4957-9a14-68f393b63312 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:17.918753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf99646-02c0-4957-9a14-68f393b63312 None admin] PATH_INFO: `/v3/projects/6061e445a046473ca280fc4eec54c559` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:17.919894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf99646-02c0-4957-9a14-68f393b63312 None admin] RBAC: Authorizing `identity:get_project(project_id=6061e445a046473ca280fc4eec54c559)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:17.921356 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf99646-02c0-4957-9a14-68f393b63312 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:17.922986 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 560/1122] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:24:17 2022] GET /identity/v3/projects/6061e445a046473ca280fc4eec54c559 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:17.928550 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5aff6-fe66-4e60-ade7-7311b94b9638 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:18.024417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5aff6-fe66-4e60-ade7-7311b94b9638 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:18.057815 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d65ecf-7975-438a-b252-a3ca5465e03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:18.058467 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d65ecf-7975-438a-b252-a3ca5465e03a None admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:18.058749 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d65ecf-7975-438a-b252-a3ca5465e03a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:18.059023 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d65ecf-7975-438a-b252-a3ca5465e03a None admin] PATH_INFO: `/v3/projects/6061e445a046473ca280fc4eec54c559/users/3a461301da714fa58c4ea1a7f94aa16c/roles/70f3e09bc103410789de8e15c81d9cd3` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:18.061193 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d65ecf-7975-438a-b252-a3ca5465e03a None admin] RBAC: Authorizing `identity:create_grant(project_id=6061e445a046473ca280fc4eec54c559, user_id=3a461301da714fa58c4ea1a7f94aa16c, role_id=70f3e09bc103410789de8e15c81d9cd3)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:18.062610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d65ecf-7975-438a-b252-a3ca5465e03a None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:18.076227 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 563/1123] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:24:17 2022] PUT /identity/v3/projects/6061e445a046473ca280fc4eec54c559/users/3a461301da714fa58c4ea1a7f94aa16c/roles/70f3e09bc103410789de8e15c81d9cd3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:24:20.040071 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e5bd2-5daa-4406-8323-ea0853caf79f None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:20.040701 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e5bd2-5daa-4406-8323-ea0853caf79f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:20.040701 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e5bd2-5daa-4406-8323-ea0853caf79f None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:20.042210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 561/1124] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:24:20.047656 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573c0f0-fdf5-471b-b04d-cc4e1fd3f613 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:20.048113 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573c0f0-fdf5-471b-b04d-cc4e1fd3f613 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:20.048222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573c0f0-fdf5-471b-b04d-cc4e1fd3f613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:20.060726 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:20.068344 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-d573c0f0-fdf5-471b-b04d-cc4e1fd3f613 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:20.070381 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d573c0f0-fdf5-471b-b04d-cc4e1fd3f613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:20.137153 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 564/1125] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:24:20.161061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e5bd2-5daa-4406-8323-ea0853caf79f None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:20.203236 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:20.261261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e5bd2-5daa-4406-8323-ea0853caf79f None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:20.293140 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7cbc62-a903-451d-9abc-9500025e178d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:20.293715 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cbc62-a903-451d-9abc-9500025e178d None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:20.293990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cbc62-a903-451d-9abc-9500025e178d None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:20.294262 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cbc62-a903-451d-9abc-9500025e178d None admin] PATH_INFO: `/v3/roles/70f3e09bc103410789de8e15c81d9cd3` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:20.295672 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7cbc62-a903-451d-9abc-9500025e178d None admin] RBAC: Authorizing `identity:get_role(role_id=70f3e09bc103410789de8e15c81d9cd3)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:20.297065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7cbc62-a903-451d-9abc-9500025e178d None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:20.298379 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 562/1126] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:24:20 2022] GET /identity/v3/roles/70f3e09bc103410789de8e15c81d9cd3 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:20.304110 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573c0f0-fdf5-471b-b04d-cc4e1fd3f613 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:20.404689 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573c0f0-fdf5-471b-b04d-cc4e1fd3f613 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:20.437857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc79c5-4a3d-4a29-a857-541342afc6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:20.438482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc79c5-4a3d-4a29-a857-541342afc6fe None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:20.438760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc79c5-4a3d-4a29-a857-541342afc6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:20.439029 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc79c5-4a3d-4a29-a857-541342afc6fe None admin] PATH_INFO: `/v3/users/3a461301da714fa58c4ea1a7f94aa16c` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:20.440158 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc79c5-4a3d-4a29-a857-541342afc6fe None admin] RBAC: Authorizing `identity:get_user(user_id=3a461301da714fa58c4ea1a7f94aa16c)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:20.441615 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc79c5-4a3d-4a29-a857-541342afc6fe None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:20.443183 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 565/1127] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:24:20 2022] GET /identity/v3/users/3a461301da714fa58c4ea1a7f94aa16c => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:20.449630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7cbc62-a903-451d-9abc-9500025e178d None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:20.544331 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7cbc62-a903-451d-9abc-9500025e178d None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:20.553091 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:20.579153 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa738e-44b9-4e42-b096-e4399e8dc24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:20.579762 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa738e-44b9-4e42-b096-e4399e8dc24c None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:20.580040 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa738e-44b9-4e42-b096-e4399e8dc24c None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:20.580311 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa738e-44b9-4e42-b096-e4399e8dc24c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:20.645073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa738e-44b9-4e42-b096-e4399e8dc24c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:20.646499 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa738e-44b9-4e42-b096-e4399e8dc24c None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:20.746148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 563/1128] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:24:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 16:24:20.752121 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc79c5-4a3d-4a29-a857-541342afc6fe None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:20.755932 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:20.850351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc79c5-4a3d-4a29-a857-541342afc6fe None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:20.883469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67437449-4ae9-4198-adbd-6b0d32ab695c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:20.884092 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67437449-4ae9-4198-adbd-6b0d32ab695c None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:20.884391 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67437449-4ae9-4198-adbd-6b0d32ab695c None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:20.884663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67437449-4ae9-4198-adbd-6b0d32ab695c None admin] PATH_INFO: `/v3/projects/6061e445a046473ca280fc4eec54c559` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:20.885821 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67437449-4ae9-4198-adbd-6b0d32ab695c None admin] RBAC: Authorizing `identity:get_project(project_id=6061e445a046473ca280fc4eec54c559)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:20.887244 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67437449-4ae9-4198-adbd-6b0d32ab695c None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:20.888871 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 566/1129] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:24:20 2022] GET /identity/v3/projects/6061e445a046473ca280fc4eec54c559 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:20.894799 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa738e-44b9-4e42-b096-e4399e8dc24c None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:20.904549 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:20.992064 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa738e-44b9-4e42-b096-e4399e8dc24c None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:21.025363 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99158d59-5c66-4b39-b6c0-02345cbd6ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:21.025964 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99158d59-5c66-4b39-b6c0-02345cbd6ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:21.026246 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99158d59-5c66-4b39-b6c0-02345cbd6ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:21.026519 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99158d59-5c66-4b39-b6c0-02345cbd6ee4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:21.027296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99158d59-5c66-4b39-b6c0-02345cbd6ee4 None admin] RBAC: Adding query filter params (role.id=70f3e09bc103410789de8e15c81d9cd3, scope.project.id=6061e445a046473ca280fc4eec54c559, user.id=3a461301da714fa58c4ea1a7f94aa16c) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:21.027603 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99158d59-5c66-4b39-b6c0-02345cbd6ee4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:21.029046 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99158d59-5c66-4b39-b6c0-02345cbd6ee4 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:21.034615 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 564/1130] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:24:20 2022] GET /identity/v3/role_assignments?user.id=3a461301da714fa58c4ea1a7f94aa16c&scope.project.id=6061e445a046473ca280fc4eec54c559&role.id=70f3e09bc103410789de8e15c81d9cd3 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:23.033123 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157fc57-6647-4716-a733-abdaef3afead None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:23.033679 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157fc57-6647-4716-a733-abdaef3afead None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:23.033679 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157fc57-6647-4716-a733-abdaef3afead None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:23.034994 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 567/1131] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:24:23.040003 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d06f0b-25d2-4cac-8d76-93041d7ae998 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:23.040968 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d06f0b-25d2-4cac-8d76-93041d7ae998 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:23.041078 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d06f0b-25d2-4cac-8d76-93041d7ae998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:23.059249 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-b5d06f0b-25d2-4cac-8d76-93041d7ae998 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:23.060432 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b5d06f0b-25d2-4cac-8d76-93041d7ae998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:23.125969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 565/1132] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:24:23.148324 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f157fc57-6647-4716-a733-abdaef3afead None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:23.218259 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:23.249421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f157fc57-6647-4716-a733-abdaef3afead None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:23.283482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ca1f5-c222-4217-ac2a-853f15168df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:23.284058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ca1f5-c222-4217-ac2a-853f15168df5 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:23.284350 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ca1f5-c222-4217-ac2a-853f15168df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:23.284623 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ca1f5-c222-4217-ac2a-853f15168df5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:23.285788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ca1f5-c222-4217-ac2a-853f15168df5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:23.287216 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ca1f5-c222-4217-ac2a-853f15168df5 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:23.288794 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 568/1133] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:24:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:23.295271 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d06f0b-25d2-4cac-8d76-93041d7ae998 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:23.297487 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:23.392428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d06f0b-25d2-4cac-8d76-93041d7ae998 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:23.425662 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76aa951-2baa-4c54-99c1-c27ce0d1e4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:23.426273 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76aa951-2baa-4c54-99c1-c27ce0d1e4f2 None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:23.426557 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76aa951-2baa-4c54-99c1-c27ce0d1e4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:23.426836 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76aa951-2baa-4c54-99c1-c27ce0d1e4f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:23.427524 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76aa951-2baa-4c54-99c1-c27ce0d1e4f2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:23.428987 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76aa951-2baa-4c54-99c1-c27ce0d1e4f2 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:23.455699 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 566/1134] 10.222.0.74 () {60 vars in 1141 bytes} [Wed Jul 27 16:24:23 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:24:25.435353 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6136fe3-a3d7-4a23-8a93-318537827c2d None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:25.435851 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6136fe3-a3d7-4a23-8a93-318537827c2d None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:25.435851 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6136fe3-a3d7-4a23-8a93-318537827c2d None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:25.437179 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 569/1135] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:24:25.442397 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78229f2e-eef4-4efb-b5c9-e2cbcb7e1b50 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:25.442803 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78229f2e-eef4-4efb-b5c9-e2cbcb7e1b50 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:25.442953 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78229f2e-eef4-4efb-b5c9-e2cbcb7e1b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:25.460829 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-78229f2e-eef4-4efb-b5c9-e2cbcb7e1b50 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:25.462036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-78229f2e-eef4-4efb-b5c9-e2cbcb7e1b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:25.509456 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:25.527277 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 567/1136] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:24:25.552311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6136fe3-a3d7-4a23-8a93-318537827c2d None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:25.655805 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6136fe3-a3d7-4a23-8a93-318537827c2d None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:25.689675 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82834f23-1f61-49cc-a4ed-6b90ecfb3bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:25.690294 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82834f23-1f61-49cc-a4ed-6b90ecfb3bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:25.690597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82834f23-1f61-49cc-a4ed-6b90ecfb3bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:25.690872 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82834f23-1f61-49cc-a4ed-6b90ecfb3bc2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:25.692054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82834f23-1f61-49cc-a4ed-6b90ecfb3bc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:25.693631 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82834f23-1f61-49cc-a4ed-6b90ecfb3bc2 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:25.695272 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 570/1137] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:24:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:25.701144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78229f2e-eef4-4efb-b5c9-e2cbcb7e1b50 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:25.795739 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78229f2e-eef4-4efb-b5c9-e2cbcb7e1b50 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:25.828367 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0297bd-3399-49c5-83a2-b2a15d3194e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:25.828994 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0297bd-3399-49c5-83a2-b2a15d3194e5 None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:25.829277 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0297bd-3399-49c5-83a2-b2a15d3194e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:25.829548 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0297bd-3399-49c5-83a2-b2a15d3194e5 None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:25.830231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0297bd-3399-49c5-83a2-b2a15d3194e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:25.831647 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0297bd-3399-49c5-83a2-b2a15d3194e5 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:25.851871 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 568/1138] 10.222.0.74 () {60 vars in 1132 bytes} [Wed Jul 27 16:24:25 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:24:27.857343 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0f7f4-f667-44aa-96cd-9a3ac9c8f731 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:27.857890 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0f7f4-f667-44aa-96cd-9a3ac9c8f731 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:27.857890 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0f7f4-f667-44aa-96cd-9a3ac9c8f731 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:27.859227 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 571/1139] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:24:27.864497 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba1713-8fc4-4dae-a9db-b01729747a7f None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:27.864874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba1713-8fc4-4dae-a9db-b01729747a7f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:27.865085 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba1713-8fc4-4dae-a9db-b01729747a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:27.883081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-2cba1713-8fc4-4dae-a9db-b01729747a7f None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:27.884924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-2cba1713-8fc4-4dae-a9db-b01729747a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:27.917534 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:27.951153 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 569/1140] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:24:27.973896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0f7f4-f667-44aa-96cd-9a3ac9c8f731 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:27.984569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:28.073712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0f7f4-f667-44aa-96cd-9a3ac9c8f731 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:28.106376 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0efb4b-e6b1-49d2-b5eb-5e390dbe3c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:28.107009 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0efb4b-e6b1-49d2-b5eb-5e390dbe3c89 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:28.107288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0efb4b-e6b1-49d2-b5eb-5e390dbe3c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:28.107609 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0efb4b-e6b1-49d2-b5eb-5e390dbe3c89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:28.117241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0efb4b-e6b1-49d2-b5eb-5e390dbe3c89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:24:28.122552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-6f0efb4b-e6b1-49d2-b5eb-5e390dbe3c89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:24:28.122552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:28.122552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:24:28.122552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:24:28.122552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:24:28.122552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:24:28.122552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:24:28.122552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:24:28.122552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:24:28.122552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:24:28.122552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:24:28.122552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:24:28.122552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:24:28.122552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:24:28.122552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:24:28.122552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:24:28.123707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:24:28.123707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:24:28.123707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:24:28.123707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:24:28.123707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:24:28.123707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:24:28.123707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:24:28.123707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:24:28.123707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:24:28.123707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:24:28.123707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:24:28.123707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:24:28.123707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:24:28.123707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:24:28.123707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:24:28.123707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:24:28.124626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:24:28.124626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:24:28.124626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:24:28.124626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:24:28.124626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:24:28.124626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:24:28.124626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:24:28.124626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:24:28.124626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:24:28.124626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:24:28.124626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:24:28.124626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:24:28.124626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 572/1141] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:24:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:24:28.129678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba1713-8fc4-4dae-a9db-b01729747a7f None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:28.228763 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba1713-8fc4-4dae-a9db-b01729747a7f None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:28.261273 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ca182-67d5-4885-9b5d-7e9521ebddf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:28.261883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ca182-67d5-4885-9b5d-7e9521ebddf4 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:28.262161 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ca182-67d5-4885-9b5d-7e9521ebddf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:28.262432 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ca182-67d5-4885-9b5d-7e9521ebddf4 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:28.263142 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ca182-67d5-4885-9b5d-7e9521ebddf4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:28.263443 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ca182-67d5-4885-9b5d-7e9521ebddf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:28.264813 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ca182-67d5-4885-9b5d-7e9521ebddf4 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:28.272596 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 570/1142] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:24:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:28.278287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0efb4b-e6b1-49d2-b5eb-5e390dbe3c89 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:28.343275 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:28.376668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0efb4b-e6b1-49d2-b5eb-5e390dbe3c89 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:28.408656 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4438044-befb-4d7d-8249-811754669495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:28.409305 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4438044-befb-4d7d-8249-811754669495 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:28.409581 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4438044-befb-4d7d-8249-811754669495 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:28.409851 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4438044-befb-4d7d-8249-811754669495 None admin] PATH_INFO: `/v3/users/d7019e0b54a240988fe444782bfeb1df` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:28.423386 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4438044-befb-4d7d-8249-811754669495 None admin] RBAC: Authorizing `identity:get_user(user_id=d7019e0b54a240988fe444782bfeb1df)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:28.424813 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4438044-befb-4d7d-8249-811754669495 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:28.426421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 573/1143] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:24:28 2022] GET /identity/v3/users/d7019e0b54a240988fe444782bfeb1df => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:28.432270 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ca182-67d5-4885-9b5d-7e9521ebddf4 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:28.463998 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:28.529098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ca182-67d5-4885-9b5d-7e9521ebddf4 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:28.560788 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f372a-503a-4511-a88a-5861a0a06da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:28.561418 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f372a-503a-4511-a88a-5861a0a06da3 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:28.561697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f372a-503a-4511-a88a-5861a0a06da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:28.561968 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f372a-503a-4511-a88a-5861a0a06da3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:28.626851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f372a-503a-4511-a88a-5861a0a06da3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:28.628223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f372a-503a-4511-a88a-5861a0a06da3 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:28.657666 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:28.723770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 571/1144] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:24:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 16:24:28.729625 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4438044-befb-4d7d-8249-811754669495 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:28.824177 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4438044-befb-4d7d-8249-811754669495 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:28.848816 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:28.857634 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5dbbaa-dc4d-4bd5-9f9d-03986379771e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:28.858252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5dbbaa-dc4d-4bd5-9f9d-03986379771e None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:28.858530 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5dbbaa-dc4d-4bd5-9f9d-03986379771e None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:28.858800 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5dbbaa-dc4d-4bd5-9f9d-03986379771e None admin] PATH_INFO: `/v3/projects/3f96420c9bd74cdaa444374e34d24b67` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:28.859915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5dbbaa-dc4d-4bd5-9f9d-03986379771e None admin] RBAC: Authorizing `identity:get_project(project_id=3f96420c9bd74cdaa444374e34d24b67)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:28.861365 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5dbbaa-dc4d-4bd5-9f9d-03986379771e None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:28.862969 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 574/1145] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:24:28 2022] GET /identity/v3/projects/3f96420c9bd74cdaa444374e34d24b67 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:28.869339 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f372a-503a-4511-a88a-5861a0a06da3 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:28.964322 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f372a-503a-4511-a88a-5861a0a06da3 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:28.992266 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:28.997603 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2be0f1-05c5-4e8b-b9e5-76953e0221b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:28.998205 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2be0f1-05c5-4e8b-b9e5-76953e0221b1 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:28.998489 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2be0f1-05c5-4e8b-b9e5-76953e0221b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:28.998761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2be0f1-05c5-4e8b-b9e5-76953e0221b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:28.999536 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2be0f1-05c5-4e8b-b9e5-76953e0221b1 None admin] RBAC: Adding query filter params (role.id=ca37e43082fa40c6a88b12680a0ca94b, scope.project.id=3f96420c9bd74cdaa444374e34d24b67, user.id=d7019e0b54a240988fe444782bfeb1df) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:28.999842 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2be0f1-05c5-4e8b-b9e5-76953e0221b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:29.001295 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2be0f1-05c5-4e8b-b9e5-76953e0221b1 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:29.006567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 572/1146] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:24:28 2022] GET /identity/v3/role_assignments?user.id=d7019e0b54a240988fe444782bfeb1df&scope.project.id=3f96420c9bd74cdaa444374e34d24b67&role.id=ca37e43082fa40c6a88b12680a0ca94b => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:24:30.960400 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4372f-165b-4af4-a4ec-5d09f9d4eb23 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:30.960995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4372f-165b-4af4-a4ec-5d09f9d4eb23 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:30.961088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4372f-165b-4af4-a4ec-5d09f9d4eb23 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:30.962363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 575/1147] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:24:30.967394 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf8626-c504-4641-9509-31e48f51390d None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:30.967826 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf8626-c504-4641-9509-31e48f51390d None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:30.967951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf8626-c504-4641-9509-31e48f51390d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:30.986206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-03cf8626-c504-4641-9509-31e48f51390d None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:30.987383 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-03cf8626-c504-4641-9509-31e48f51390d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:31.051317 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 573/1148] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:24:31.074205 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4372f-165b-4af4-a4ec-5d09f9d4eb23 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:31.172299 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4372f-165b-4af4-a4ec-5d09f9d4eb23 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:31.205453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2b3a4-b435-40b9-ac31-193e3a148429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:31.206076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2b3a4-b435-40b9-ac31-193e3a148429 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:31.206354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2b3a4-b435-40b9-ac31-193e3a148429 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:31.206626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2b3a4-b435-40b9-ac31-193e3a148429 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:31.216456 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2b3a4-b435-40b9-ac31-193e3a148429 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:24:31.221249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-0fe2b3a4-b435-40b9-ac31-193e3a148429 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:24:31.221249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:31.221249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:24:31.221249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:24:31.221249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:24:31.221249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:24:31.221249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:24:31.221249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:24:31.221249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:24:31.221249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:24:31.221249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:24:31.221249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:24:31.221249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:24:31.221249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:24:31.221249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:24:31.221249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:24:31.222173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:24:31.222173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:24:31.222173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:24:31.222173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:24:31.222173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:24:31.222173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:24:31.222173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:24:31.222173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:24:31.222173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:24:31.222173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:24:31.222173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:24:31.222173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:24:31.222173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:24:31.222173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:24:31.222173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:24:31.222173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:24:31.223088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:24:31.223088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:24:31.223088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:24:31.223088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:24:31.223088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:24:31.223088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:24:31.223088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:24:31.223088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:24:31.223088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:24:31.223088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:24:31.223088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:24:31.223088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:24:31.223088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 576/1149] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:24:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:24:31.228910 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf8626-c504-4641-9509-31e48f51390d None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:31.328206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf8626-c504-4641-9509-31e48f51390d None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:31.352322 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:31.361724 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca307fd-6ed7-4211-8e8b-bca228c9731a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:31.362334 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca307fd-6ed7-4211-8e8b-bca228c9731a None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:31.362617 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca307fd-6ed7-4211-8e8b-bca228c9731a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:31.362892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca307fd-6ed7-4211-8e8b-bca228c9731a None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:31.363606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca307fd-6ed7-4211-8e8b-bca228c9731a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:31.363912 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca307fd-6ed7-4211-8e8b-bca228c9731a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:31.365309 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca307fd-6ed7-4211-8e8b-bca228c9731a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:31.373676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 574/1150] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:24:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:31.379347 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2b3a4-b435-40b9-ac31-193e3a148429 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:31.433712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:31.479603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2b3a4-b435-40b9-ac31-193e3a148429 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:31.513121 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10111d-8986-4e3e-a836-252d7f122b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:31.513747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10111d-8986-4e3e-a836-252d7f122b74 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:31.514027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10111d-8986-4e3e-a836-252d7f122b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:31.514299 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10111d-8986-4e3e-a836-252d7f122b74 None admin] PATH_INFO: `/v3/users/d7019e0b54a240988fe444782bfeb1df` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:31.515442 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10111d-8986-4e3e-a836-252d7f122b74 None admin] RBAC: Authorizing `identity:get_user(user_id=d7019e0b54a240988fe444782bfeb1df)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:31.516870 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10111d-8986-4e3e-a836-252d7f122b74 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:31.518473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 577/1151] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:24:31 2022] GET /identity/v3/users/d7019e0b54a240988fe444782bfeb1df => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:31.524187 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca307fd-6ed7-4211-8e8b-bca228c9731a None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:31.620131 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca307fd-6ed7-4211-8e8b-bca228c9731a None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:31.652239 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ecddb-97b3-49fb-8e28-c3feec01c471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:31.652863 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ecddb-97b3-49fb-8e28-c3feec01c471 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:31.653172 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ecddb-97b3-49fb-8e28-c3feec01c471 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:31.653445 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ecddb-97b3-49fb-8e28-c3feec01c471 None admin] PATH_INFO: `/v3/projects/3f96420c9bd74cdaa444374e34d24b67` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:31.654601 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ecddb-97b3-49fb-8e28-c3feec01c471 None admin] RBAC: Authorizing `identity:get_project(project_id=3f96420c9bd74cdaa444374e34d24b67)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:31.656029 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ecddb-97b3-49fb-8e28-c3feec01c471 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:31.657689 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 575/1152] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:24:31 2022] GET /identity/v3/projects/3f96420c9bd74cdaa444374e34d24b67 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:31.663441 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10111d-8986-4e3e-a836-252d7f122b74 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:31.759809 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10111d-8986-4e3e-a836-252d7f122b74 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:31.781323 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:31.792756 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab98d73-5b68-4942-9462-50c20dc1b009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:31.793433 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab98d73-5b68-4942-9462-50c20dc1b009 None admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:31.793714 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab98d73-5b68-4942-9462-50c20dc1b009 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:31.793990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab98d73-5b68-4942-9462-50c20dc1b009 None admin] PATH_INFO: `/v3/projects/3f96420c9bd74cdaa444374e34d24b67/users/d7019e0b54a240988fe444782bfeb1df/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:31.795893 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab98d73-5b68-4942-9462-50c20dc1b009 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f96420c9bd74cdaa444374e34d24b67, user_id=d7019e0b54a240988fe444782bfeb1df, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:31.797345 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab98d73-5b68-4942-9462-50c20dc1b009 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:31.810248 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 578/1153] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:24:31 2022] PUT /identity/v3/projects/3f96420c9bd74cdaa444374e34d24b67/users/d7019e0b54a240988fe444782bfeb1df/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:24:33.759765 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae721c0-db36-4333-b610-8f99b32bde12 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:33.760287 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae721c0-db36-4333-b610-8f99b32bde12 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:33.760398 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae721c0-db36-4333-b610-8f99b32bde12 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:33.761721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 576/1154] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:24:33.766686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4c490-d02f-430d-9ca7-9da4ab829b28 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:33.767100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4c490-d02f-430d-9ca7-9da4ab829b28 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:33.767237 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4c490-d02f-430d-9ca7-9da4ab829b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:33.785440 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-97b4c490-d02f-430d-9ca7-9da4ab829b28 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:33.786609 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-97b4c490-d02f-430d-9ca7-9da4ab829b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:33.860971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 579/1155] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:24:33.883356 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae721c0-db36-4333-b610-8f99b32bde12 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:33.938899 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:33.982740 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae721c0-db36-4333-b610-8f99b32bde12 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:34.014359 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81eed6-be94-404e-992a-9c8b3444efb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:34.014936 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81eed6-be94-404e-992a-9c8b3444efb2 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:34.015217 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81eed6-be94-404e-992a-9c8b3444efb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:34.015491 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81eed6-be94-404e-992a-9c8b3444efb2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:34.026250 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f81eed6-be94-404e-992a-9c8b3444efb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:24:34.031133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-5f81eed6-be94-404e-992a-9c8b3444efb2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:24:34.031133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:34.031133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:24:34.031133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:24:34.031133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:24:34.031133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:24:34.031133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:24:34.031133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:24:34.031133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:24:34.031133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:24:34.031133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:24:34.031133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:24:34.031133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:24:34.031133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:24:34.031133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:24:34.031133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:24:34.032203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:24:34.032203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:24:34.032203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:24:34.032203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:24:34.032203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:24:34.032203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:24:34.032203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:24:34.032203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:24:34.032203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:24:34.032203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:24:34.032203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:24:34.032203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:24:34.032203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:24:34.032203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:24:34.032203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:24:34.032203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:24:34.033112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:24:34.033112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:24:34.033112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:24:34.033112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:24:34.033112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:24:34.033112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:24:34.033112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:24:34.033112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:24:34.033112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:24:34.033112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:24:34.033112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:24:34.033112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:24:34.033112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 577/1156] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:24:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:24:34.038200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b4c490-d02f-430d-9ca7-9da4ab829b28 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:34.135651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b4c490-d02f-430d-9ca7-9da4ab829b28 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:34.148425 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:34.172016 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7fd326-9b59-488a-85cd-9ad2107401c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:34.172931 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fd326-9b59-488a-85cd-9ad2107401c8 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:34.173363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fd326-9b59-488a-85cd-9ad2107401c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:34.173771 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fd326-9b59-488a-85cd-9ad2107401c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:34.174853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7fd326-9b59-488a-85cd-9ad2107401c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:34.175311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7fd326-9b59-488a-85cd-9ad2107401c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:34.177540 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7fd326-9b59-488a-85cd-9ad2107401c8 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:34.189655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 580/1157] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:24:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:34.195284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81eed6-be94-404e-992a-9c8b3444efb2 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:34.301850 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81eed6-be94-404e-992a-9c8b3444efb2 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:34.317030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:34.335145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da3551-ecec-47ab-84a5-5e92ca0a1758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:34.335779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da3551-ecec-47ab-84a5-5e92ca0a1758 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:34.336061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da3551-ecec-47ab-84a5-5e92ca0a1758 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:34.336337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da3551-ecec-47ab-84a5-5e92ca0a1758 None admin] PATH_INFO: `/v3/users/d7019e0b54a240988fe444782bfeb1df` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:34.338078 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da3551-ecec-47ab-84a5-5e92ca0a1758 None admin] RBAC: Authorizing `identity:get_user(user_id=d7019e0b54a240988fe444782bfeb1df)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:34.339517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da3551-ecec-47ab-84a5-5e92ca0a1758 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:34.341141 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 578/1158] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:24:34 2022] GET /identity/v3/users/d7019e0b54a240988fe444782bfeb1df => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:34.346854 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7fd326-9b59-488a-85cd-9ad2107401c8 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:34.453000 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7fd326-9b59-488a-85cd-9ad2107401c8 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:34.485417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4644b0e-0971-41cb-b4aa-c6fa40ea5de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:34.486016 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4644b0e-0971-41cb-b4aa-c6fa40ea5de1 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:34.486297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4644b0e-0971-41cb-b4aa-c6fa40ea5de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:34.486571 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4644b0e-0971-41cb-b4aa-c6fa40ea5de1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:34.527916 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:34.556840 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4644b0e-0971-41cb-b4aa-c6fa40ea5de1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:34.559039 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4644b0e-0971-41cb-b4aa-c6fa40ea5de1 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:34.667005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 581/1159] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:24:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 16:24:34.672881 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da3551-ecec-47ab-84a5-5e92ca0a1758 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:34.769639 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da3551-ecec-47ab-84a5-5e92ca0a1758 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:34.801138 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb685e-8bb6-4aed-886d-e8b64f10e5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:34.801752 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb685e-8bb6-4aed-886d-e8b64f10e5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:34.802029 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb685e-8bb6-4aed-886d-e8b64f10e5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:34.802298 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb685e-8bb6-4aed-886d-e8b64f10e5e8 None admin] PATH_INFO: `/v3/projects/3f96420c9bd74cdaa444374e34d24b67` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:34.803416 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb685e-8bb6-4aed-886d-e8b64f10e5e8 None admin] RBAC: Authorizing `identity:get_project(project_id=3f96420c9bd74cdaa444374e34d24b67)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:34.804838 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb685e-8bb6-4aed-886d-e8b64f10e5e8 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:34.807210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 579/1160] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:24:34 2022] GET /identity/v3/projects/3f96420c9bd74cdaa444374e34d24b67 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:34.813034 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4644b0e-0971-41cb-b4aa-c6fa40ea5de1 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:34.886679 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:34.911058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4644b0e-0971-41cb-b4aa-c6fa40ea5de1 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:34.944649 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08842d-1968-462b-8283-c6302f9f0168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:34.945282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08842d-1968-462b-8283-c6302f9f0168 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:34.945564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08842d-1968-462b-8283-c6302f9f0168 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:34.945837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08842d-1968-462b-8283-c6302f9f0168 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:34.946653 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08842d-1968-462b-8283-c6302f9f0168 None admin] RBAC: Adding query filter params (role.id=ca37e43082fa40c6a88b12680a0ca94b, scope.project.id=3f96420c9bd74cdaa444374e34d24b67, user.id=d7019e0b54a240988fe444782bfeb1df) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:34.946959 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08842d-1968-462b-8283-c6302f9f0168 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:34.948378 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08842d-1968-462b-8283-c6302f9f0168 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:34.954280 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 582/1161] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:24:34 2022] GET /identity/v3/role_assignments?user.id=d7019e0b54a240988fe444782bfeb1df&scope.project.id=3f96420c9bd74cdaa444374e34d24b67&role.id=ca37e43082fa40c6a88b12680a0ca94b => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:36.932342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f5272-6890-43ca-a8d3-5417be1c499d None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:36.932968 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f5272-6890-43ca-a8d3-5417be1c499d None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:36.932968 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f5272-6890-43ca-a8d3-5417be1c499d None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:36.934367 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 580/1162] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:24:36.939362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ad94d-936f-4a8c-ad40-327466110ed7 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:36.939798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ad94d-936f-4a8c-ad40-327466110ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:36.939909 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ad94d-936f-4a8c-ad40-327466110ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:36.957899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-9a0ad94d-936f-4a8c-ad40-327466110ed7 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:36.959077 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-9a0ad94d-936f-4a8c-ad40-327466110ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:37.023915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 583/1163] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:24:37.046496 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f5272-6890-43ca-a8d3-5417be1c499d None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:37.099056 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:37.145668 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f5272-6890-43ca-a8d3-5417be1c499d None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:37.178632 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3ce75-7f5b-48c7-8f78-55fc295d9ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:37.179250 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3ce75-7f5b-48c7-8f78-55fc295d9ecd None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:37.179532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3ce75-7f5b-48c7-8f78-55fc295d9ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:37.179809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3ce75-7f5b-48c7-8f78-55fc295d9ecd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:37.244960 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3ce75-7f5b-48c7-8f78-55fc295d9ecd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:37.246417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3ce75-7f5b-48c7-8f78-55fc295d9ecd None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:37.295954 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:37.342166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 581/1164] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:24:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 16:24:37.348164 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ad94d-936f-4a8c-ad40-327466110ed7 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:37.415672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:37.448140 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ad94d-936f-4a8c-ad40-327466110ed7 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:37.481789 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db82dd-cf7e-4398-bb98-627c64682019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:37.482416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db82dd-cf7e-4398-bb98-627c64682019 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:37.482698 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db82dd-cf7e-4398-bb98-627c64682019 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:37.482976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db82dd-cf7e-4398-bb98-627c64682019 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:37.489470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db82dd-cf7e-4398-bb98-627c64682019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:37.490917 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db82dd-cf7e-4398-bb98-627c64682019 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:37.502774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-84db82dd-cf7e-4398-bb98-627c64682019 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 27 16:24:37.502774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:37.502774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 27 16:24:37.502774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 27 16:24:37.502774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:24:37.502774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:24:37.502774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:24:37.502774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:24:37.502774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 27 16:24:37.502774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:24:37.502774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:24:37.502774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:24:37.502774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:37.502774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:24:37.502774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:24:37.502774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:24:37.502774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:24:37.503903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:24:37.503903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:24:37.503903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:24:37.503903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:24:37.503903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:24:37.503903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:24:37.503903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:24:37.503903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:24:37.503903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:24:37.503903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:24:37.503903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:24:37.503903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:24:37.503903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:24:37.503903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:24:37.503903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:24:37.503903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:24:37.505100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:24:37.505100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:24:37.505100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:24:37.505100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:24:37.505100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:24:37.505100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:24:37.505100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:24:37.505100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:24:37.505100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:24:37.505100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:24:37.505100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 27 16:24:37.505100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:24:37.505100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 27 16:24:37.505100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:24:37.505100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 584/1165] 10.222.0.74 () {56 vars in 1117 bytes} [Wed Jul 27 16:24:37 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:24:37.510019 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3ce75-7f5b-48c7-8f78-55fc295d9ecd None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:37.608048 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3ce75-7f5b-48c7-8f78-55fc295d9ecd None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:37.640058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80154400-9f86-428f-8ad7-9e9a08c9e75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:37.640666 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80154400-9f86-428f-8ad7-9e9a08c9e75c None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:37.640967 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80154400-9f86-428f-8ad7-9e9a08c9e75c None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:37.641242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80154400-9f86-428f-8ad7-9e9a08c9e75c None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:37.641920 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80154400-9f86-428f-8ad7-9e9a08c9e75c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:37.642224 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80154400-9f86-428f-8ad7-9e9a08c9e75c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:37.643581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80154400-9f86-428f-8ad7-9e9a08c9e75c None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:37.650962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 582/1166] 10.222.0.74 () {56 vars in 1115 bytes} [Wed Jul 27 16:24:37 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:24:39.599279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0eda40-5ed7-4d7f-8e08-ecd40e290aa1 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:39.599863 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0eda40-5ed7-4d7f-8e08-ecd40e290aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:39.599863 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0eda40-5ed7-4d7f-8e08-ecd40e290aa1 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:39.601206 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 585/1167] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:24:39.606878 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82791912-f433-4bf0-b0e8-4f9766dc4b23 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:39.607323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82791912-f433-4bf0-b0e8-4f9766dc4b23 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:39.607472 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82791912-f433-4bf0-b0e8-4f9766dc4b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:39.625960 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-82791912-f433-4bf0-b0e8-4f9766dc4b23 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:39.627127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-82791912-f433-4bf0-b0e8-4f9766dc4b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:39.637895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:39.692071 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 583/1168] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 16:24:39.714256 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0eda40-5ed7-4d7f-8e08-ecd40e290aa1 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:39.811686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0eda40-5ed7-4d7f-8e08-ecd40e290aa1 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:39.837353 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:39.845200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccce327-46e6-4d04-befa-6b7600971624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:39.845800 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccce327-46e6-4d04-befa-6b7600971624 None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:39.846082 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccce327-46e6-4d04-befa-6b7600971624 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:39.846357 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccce327-46e6-4d04-befa-6b7600971624 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:39.847051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccce327-46e6-4d04-befa-6b7600971624 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:39.848398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccce327-46e6-4d04-befa-6b7600971624 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:39.863571 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 586/1169] 10.222.0.74 () {60 vars in 1138 bytes} [Wed Jul 27 16:24:39 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:24:41.865921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba598faa-4693-4cc1-a567-2c2f43ce4055 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:41.866868 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba598faa-4693-4cc1-a567-2c2f43ce4055 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:41.866868 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba598faa-4693-4cc1-a567-2c2f43ce4055 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:41.867853 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 584/1170] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:24:41.874037 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d4e3a-d397-408d-a04d-8c764213c064 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:41.874523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d4e3a-d397-408d-a04d-8c764213c064 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:41.874625 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d4e3a-d397-408d-a04d-8c764213c064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:41.895750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-2a8d4e3a-d397-408d-a04d-8c764213c064 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:41.896966 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-2a8d4e3a-d397-408d-a04d-8c764213c064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:41.961545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 587/1171] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:24:41.984494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba598faa-4693-4cc1-a567-2c2f43ce4055 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:42.082176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba598faa-4693-4cc1-a567-2c2f43ce4055 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:42.115138 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78a445-3d0e-43b7-8e96-c15504efd826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:42.115790 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78a445-3d0e-43b7-8e96-c15504efd826 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:42.116119 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78a445-3d0e-43b7-8e96-c15504efd826 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:42.116345 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78a445-3d0e-43b7-8e96-c15504efd826 None admin] PATH_INFO: `/v3/domains/831d8ee56ee1476ebff27e6f21b3d978` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:42.126661 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78a445-3d0e-43b7-8e96-c15504efd826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=831d8ee56ee1476ebff27e6f21b3d978)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:42.128101 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78a445-3d0e-43b7-8e96-c15504efd826 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:42.129708 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 585/1172] 10.222.0.74 () {56 vars in 1183 bytes} [Wed Jul 27 16:24:41 2022] GET /identity/v3/domains/831d8ee56ee1476ebff27e6f21b3d978 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:42.135335 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d4e3a-d397-408d-a04d-8c764213c064 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:42.232544 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d4e3a-d397-408d-a04d-8c764213c064 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:42.249011 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:42.265930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26947e57-c82c-4b1c-8771-7221c9717706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:42.266538 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26947e57-c82c-4b1c-8771-7221c9717706 None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:42.266818 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26947e57-c82c-4b1c-8771-7221c9717706 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:42.267091 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26947e57-c82c-4b1c-8771-7221c9717706 None admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:42.267757 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26947e57-c82c-4b1c-8771-7221c9717706 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:42.269211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26947e57-c82c-4b1c-8771-7221c9717706 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:42.294409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 588/1173] 10.222.0.74 () {60 vars in 1141 bytes} [Wed Jul 27 16:24:42 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:24:44.299162 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c9ea9-b84b-4dd4-946c-4dacfadae2d3 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:44.299683 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c9ea9-b84b-4dd4-946c-4dacfadae2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:44.299797 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c9ea9-b84b-4dd4-946c-4dacfadae2d3 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:44.301089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 586/1174] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:24:44.306613 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754d18f-3c92-4fd2-aea3-d8345094f1cc None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:44.307004 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754d18f-3c92-4fd2-aea3-d8345094f1cc None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:44.307177 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754d18f-3c92-4fd2-aea3-d8345094f1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:44.325419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-d754d18f-3c92-4fd2-aea3-d8345094f1cc None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:44.326848 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d754d18f-3c92-4fd2-aea3-d8345094f1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:44.391601 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 589/1175] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:24:44.415344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c9ea9-b84b-4dd4-946c-4dacfadae2d3 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:44.425298 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:44.516385 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c9ea9-b84b-4dd4-946c-4dacfadae2d3 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:44.548748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1db4f-65fa-4f9b-9655-fb56c3dbb2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:44.549403 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1db4f-65fa-4f9b-9655-fb56c3dbb2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:44.549688 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1db4f-65fa-4f9b-9655-fb56c3dbb2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:44.549965 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1db4f-65fa-4f9b-9655-fb56c3dbb2d8 None admin] PATH_INFO: `/v3/domains/831d8ee56ee1476ebff27e6f21b3d978` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:44.551111 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1db4f-65fa-4f9b-9655-fb56c3dbb2d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=831d8ee56ee1476ebff27e6f21b3d978)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:44.552550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1db4f-65fa-4f9b-9655-fb56c3dbb2d8 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:44.554161 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 587/1176] 10.222.0.74 () {56 vars in 1183 bytes} [Wed Jul 27 16:24:44 2022] GET /identity/v3/domains/831d8ee56ee1476ebff27e6f21b3d978 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:44.560160 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754d18f-3c92-4fd2-aea3-d8345094f1cc None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:44.652118 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:44.655014 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754d18f-3c92-4fd2-aea3-d8345094f1cc None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:44.689687 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61066e-7309-41ea-b48e-900dad4609fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:44.690241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61066e-7309-41ea-b48e-900dad4609fe None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:44.690523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61066e-7309-41ea-b48e-900dad4609fe None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:44.690805 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61066e-7309-41ea-b48e-900dad4609fe None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:44.691477 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61066e-7309-41ea-b48e-900dad4609fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:44.692928 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61066e-7309-41ea-b48e-900dad4609fe None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:44.713984 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 590/1177] 10.222.0.74 () {60 vars in 1132 bytes} [Wed Jul 27 16:24:44 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:24:46.705816 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221eab4-03b1-4db0-b62a-e43a31dde1f2 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:46.706371 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221eab4-03b1-4db0-b62a-e43a31dde1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:46.706371 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221eab4-03b1-4db0-b62a-e43a31dde1f2 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:46.707701 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 588/1178] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:24:46.713785 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b90d7e-815a-4ae4-ba6e-dd910164895a None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:46.714177 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b90d7e-815a-4ae4-ba6e-dd910164895a None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:46.714332 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b90d7e-815a-4ae4-ba6e-dd910164895a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:46.733360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-d8b90d7e-815a-4ae4-ba6e-dd910164895a None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:46.734536 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d8b90d7e-815a-4ae4-ba6e-dd910164895a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:46.800371 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 591/1179] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:24:46.822874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221eab4-03b1-4db0-b62a-e43a31dde1f2 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:46.897802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:46.927736 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221eab4-03b1-4db0-b62a-e43a31dde1f2 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:46.960232 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c27ea7-658e-4f57-8f07-0ade1b4af7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:46.960801 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c27ea7-658e-4f57-8f07-0ade1b4af7be None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:46.961108 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c27ea7-658e-4f57-8f07-0ade1b4af7be None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:46.961384 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c27ea7-658e-4f57-8f07-0ade1b4af7be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:46.972273 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c27ea7-658e-4f57-8f07-0ade1b4af7be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:24:46.977214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-80c27ea7-658e-4f57-8f07-0ade1b4af7be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:24:46.977214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:46.977214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:24:46.977214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:24:46.977214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:24:46.977214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:24:46.977214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:24:46.977214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:24:46.977214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:24:46.977214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:24:46.977214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:24:46.977214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:24:46.977214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:24:46.977214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:24:46.977214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:24:46.977214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:24:46.978167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:24:46.978167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:24:46.978167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:24:46.978167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:24:46.978167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:24:46.978167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:24:46.978167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:24:46.978167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:24:46.978167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:24:46.978167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:24:46.978167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:24:46.978167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:24:46.978167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:24:46.978167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:24:46.978167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:24:46.978167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:24:46.979123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:24:46.979123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:24:46.979123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:24:46.979123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:24:46.979123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:24:46.979123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:24:46.979123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:24:46.979123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:24:46.979123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:24:46.979123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:24:46.979123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:24:46.979123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:24:46.979123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 589/1180] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:24:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:24:46.984370 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b90d7e-815a-4ae4-ba6e-dd910164895a None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:47.064775 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:47.080812 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b90d7e-815a-4ae4-ba6e-dd910164895a None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:47.113416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a749f8ad-d697-4938-ac0a-efe5c4e2d346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:47.114020 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749f8ad-d697-4938-ac0a-efe5c4e2d346 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:47.114301 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749f8ad-d697-4938-ac0a-efe5c4e2d346 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:47.114574 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749f8ad-d697-4938-ac0a-efe5c4e2d346 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:47.115280 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a749f8ad-d697-4938-ac0a-efe5c4e2d346 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:47.115582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a749f8ad-d697-4938-ac0a-efe5c4e2d346 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:47.116961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a749f8ad-d697-4938-ac0a-efe5c4e2d346 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:47.124688 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 592/1181] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:24:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:24:47.130238 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c27ea7-658e-4f57-8f07-0ade1b4af7be None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:47.228180 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c27ea7-658e-4f57-8f07-0ade1b4af7be None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:47.260705 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49802c92-1edf-4d99-8ecc-105ba107a64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:47.261357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49802c92-1edf-4d99-8ecc-105ba107a64e None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:47.261642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49802c92-1edf-4d99-8ecc-105ba107a64e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:47.261916 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49802c92-1edf-4d99-8ecc-105ba107a64e None admin] PATH_INFO: `/v3/users/d7e07cb9e74d4bed8c3e9671c76dbc0c` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:47.263779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:47.275916 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49802c92-1edf-4d99-8ecc-105ba107a64e None admin] RBAC: Authorizing `identity:get_user(user_id=d7e07cb9e74d4bed8c3e9671c76dbc0c)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:47.277389 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49802c92-1edf-4d99-8ecc-105ba107a64e None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:47.278997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 590/1182] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:24:47 2022] GET /identity/v3/users/d7e07cb9e74d4bed8c3e9671c76dbc0c => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:47.284823 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a749f8ad-d697-4938-ac0a-efe5c4e2d346 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:47.388469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a749f8ad-d697-4938-ac0a-efe5c4e2d346 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:47.420719 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab99704-c290-4015-b026-68d9afdae91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:47.421356 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab99704-c290-4015-b026-68d9afdae91b None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:47.421634 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab99704-c290-4015-b026-68d9afdae91b None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:47.421906 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab99704-c290-4015-b026-68d9afdae91b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:47.439187 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:47.488142 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab99704-c290-4015-b026-68d9afdae91b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:47.489551 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab99704-c290-4015-b026-68d9afdae91b None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:47.589682 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 593/1183] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:24:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 16:24:47.595544 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49802c92-1edf-4d99-8ecc-105ba107a64e None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:47.700792 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49802c92-1edf-4d99-8ecc-105ba107a64e None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:47.734447 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebaa63-d621-4345-8735-b34e35051873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:47.735079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebaa63-d621-4345-8735-b34e35051873 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:47.735356 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebaa63-d621-4345-8735-b34e35051873 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:47.735627 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebaa63-d621-4345-8735-b34e35051873 None admin] PATH_INFO: `/v3/projects/ec38d216b5544ec2857365422e6f7408` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:47.736753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebaa63-d621-4345-8735-b34e35051873 None admin] RBAC: Authorizing `identity:get_project(project_id=ec38d216b5544ec2857365422e6f7408)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:47.738262 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebaa63-d621-4345-8735-b34e35051873 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:47.739852 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 591/1184] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:24:47 2022] GET /identity/v3/projects/ec38d216b5544ec2857365422e6f7408 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:47.746090 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab99704-c290-4015-b026-68d9afdae91b None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:47.794037 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:47.844500 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab99704-c290-4015-b026-68d9afdae91b None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:47.876841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee7c1c-3872-4d02-9dd5-1e08542ae324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:47.877470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee7c1c-3872-4d02-9dd5-1e08542ae324 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:47.877750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee7c1c-3872-4d02-9dd5-1e08542ae324 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:47.878021 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee7c1c-3872-4d02-9dd5-1e08542ae324 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:47.878799 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee7c1c-3872-4d02-9dd5-1e08542ae324 None admin] RBAC: Adding query filter params (role.id=ca37e43082fa40c6a88b12680a0ca94b, scope.project.id=ec38d216b5544ec2857365422e6f7408, user.id=d7e07cb9e74d4bed8c3e9671c76dbc0c) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:47.879105 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee7c1c-3872-4d02-9dd5-1e08542ae324 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:47.880515 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee7c1c-3872-4d02-9dd5-1e08542ae324 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:47.886250 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 594/1185] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:24:47 2022] GET /identity/v3/role_assignments?user.id=d7e07cb9e74d4bed8c3e9671c76dbc0c&scope.project.id=ec38d216b5544ec2857365422e6f7408&role.id=ca37e43082fa40c6a88b12680a0ca94b => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:24:49.836356 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7af4b-cf2b-4e4a-9220-a8f7a443fbd3 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:49.837465 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7af4b-cf2b-4e4a-9220-a8f7a443fbd3 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:49.837465 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7af4b-cf2b-4e4a-9220-a8f7a443fbd3 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:49.838768 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 592/1186] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:24:49.844001 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc0d8b-87a2-48ff-b381-8b5ecd75a1d5 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:49.844445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc0d8b-87a2-48ff-b381-8b5ecd75a1d5 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:49.844545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc0d8b-87a2-48ff-b381-8b5ecd75a1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:49.862978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-91dc0d8b-87a2-48ff-b381-8b5ecd75a1d5 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:49.864150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-91dc0d8b-87a2-48ff-b381-8b5ecd75a1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:49.929172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 595/1187] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 16:24:49.950987 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7af4b-cf2b-4e4a-9220-a8f7a443fbd3 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:50.025203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:50.072293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7af4b-cf2b-4e4a-9220-a8f7a443fbd3 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:50.105442 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450818c-395a-486d-a868-224a17dccf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:50.106040 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450818c-395a-486d-a868-224a17dccf4a None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:50.106322 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450818c-395a-486d-a868-224a17dccf4a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:50.106597 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450818c-395a-486d-a868-224a17dccf4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:50.116338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450818c-395a-486d-a868-224a17dccf4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:24:50.121278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-5450818c-395a-486d-a868-224a17dccf4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:24:50.121278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:50.121278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:24:50.121278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:24:50.121278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:24:50.121278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:24:50.121278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:24:50.121278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:24:50.121278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:24:50.121278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:24:50.121278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:24:50.121278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:24:50.121278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:24:50.121278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:24:50.121278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:24:50.121278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:24:50.122413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:24:50.122413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:24:50.122413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:24:50.122413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:24:50.122413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:24:50.122413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:24:50.122413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:24:50.122413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:24:50.122413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:24:50.122413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:24:50.122413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:24:50.122413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:24:50.122413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:24:50.122413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:24:50.122413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:24:50.122413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:24:50.123347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:24:50.123347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:24:50.123347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:24:50.123347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:24:50.123347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:24:50.123347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:24:50.123347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:24:50.123347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:24:50.123347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:24:50.123347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:24:50.123347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:24:50.123347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:24:50.123347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 593/1188] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:24:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:24:50.128406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc0d8b-87a2-48ff-b381-8b5ecd75a1d5 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:50.173768 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:50.227891 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc0d8b-87a2-48ff-b381-8b5ecd75a1d5 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:50.260795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452983f1-b9cd-4410-aa8e-c2ca7906be01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:50.261421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452983f1-b9cd-4410-aa8e-c2ca7906be01 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:50.261699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452983f1-b9cd-4410-aa8e-c2ca7906be01 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:50.261970 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452983f1-b9cd-4410-aa8e-c2ca7906be01 None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:50.262672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452983f1-b9cd-4410-aa8e-c2ca7906be01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:50.262970 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452983f1-b9cd-4410-aa8e-c2ca7906be01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:50.264324 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452983f1-b9cd-4410-aa8e-c2ca7906be01 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:50.271993 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 596/1189] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:24:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:50.277680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450818c-395a-486d-a868-224a17dccf4a None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:50.379046 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450818c-395a-486d-a868-224a17dccf4a None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:50.398449 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:50.412479 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da89ee-a68d-4526-843b-aabe2d32b89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:50.413146 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da89ee-a68d-4526-843b-aabe2d32b89d None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:50.413429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da89ee-a68d-4526-843b-aabe2d32b89d None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:50.413721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da89ee-a68d-4526-843b-aabe2d32b89d None admin] PATH_INFO: `/v3/users/d7e07cb9e74d4bed8c3e9671c76dbc0c` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:50.414897 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da89ee-a68d-4526-843b-aabe2d32b89d None admin] RBAC: Authorizing `identity:get_user(user_id=d7e07cb9e74d4bed8c3e9671c76dbc0c)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:50.416333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da89ee-a68d-4526-843b-aabe2d32b89d None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:50.417960 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 594/1190] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:24:50 2022] GET /identity/v3/users/d7e07cb9e74d4bed8c3e9671c76dbc0c => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:50.423705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452983f1-b9cd-4410-aa8e-c2ca7906be01 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:50.520333 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452983f1-b9cd-4410-aa8e-c2ca7906be01 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:50.522603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:50.553338 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b05f52-0849-45d2-ad94-a075f86f5fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:50.553970 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b05f52-0849-45d2-ad94-a075f86f5fcd None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:50.554250 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b05f52-0849-45d2-ad94-a075f86f5fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:50.554525 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b05f52-0849-45d2-ad94-a075f86f5fcd None admin] PATH_INFO: `/v3/projects/ec38d216b5544ec2857365422e6f7408` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:50.555764 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b05f52-0849-45d2-ad94-a075f86f5fcd None admin] RBAC: Authorizing `identity:get_project(project_id=ec38d216b5544ec2857365422e6f7408)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:50.557219 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b05f52-0849-45d2-ad94-a075f86f5fcd None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:50.558859 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 597/1191] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:24:50 2022] GET /identity/v3/projects/ec38d216b5544ec2857365422e6f7408 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:50.564864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da89ee-a68d-4526-843b-aabe2d32b89d None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:50.666067 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da89ee-a68d-4526-843b-aabe2d32b89d None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:50.708738 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ab73c-e513-4ada-bdb4-207ab55bdab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:50.709445 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ab73c-e513-4ada-bdb4-207ab55bdab8 None admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:50.709753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ab73c-e513-4ada-bdb4-207ab55bdab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:50.710054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ab73c-e513-4ada-bdb4-207ab55bdab8 None admin] PATH_INFO: `/v3/projects/ec38d216b5544ec2857365422e6f7408/users/d7e07cb9e74d4bed8c3e9671c76dbc0c/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:50.713660 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ab73c-e513-4ada-bdb4-207ab55bdab8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec38d216b5544ec2857365422e6f7408, user_id=d7e07cb9e74d4bed8c3e9671c76dbc0c, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:50.715762 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ab73c-e513-4ada-bdb4-207ab55bdab8 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:50.733019 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 595/1192] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:24:50 2022] PUT /identity/v3/projects/ec38d216b5544ec2857365422e6f7408/users/d7e07cb9e74d4bed8c3e9671c76dbc0c/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:24:52.702868 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b440c-e092-4022-aa06-c83792b1a97f None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:52.703442 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b440c-e092-4022-aa06-c83792b1a97f None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:52.703442 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b440c-e092-4022-aa06-c83792b1a97f None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:52.704753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 598/1193] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:24:52.709895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc29a5-0890-4023-b4d5-80b66f7ff89b None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:52.710310 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc29a5-0890-4023-b4d5-80b66f7ff89b None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:52.710450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc29a5-0890-4023-b4d5-80b66f7ff89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:52.730387 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-66bc29a5-0890-4023-b4d5-80b66f7ff89b None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:24:52.731583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-66bc29a5-0890-4023-b4d5-80b66f7ff89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:24:52.776390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:52.801251 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 596/1194] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:24:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:24:52.825610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b440c-e092-4022-aa06-c83792b1a97f None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:52.932730 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b440c-e092-4022-aa06-c83792b1a97f None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:52.966434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16c830-3937-4c57-a967-cdb1b6c57cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:52.966999 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16c830-3937-4c57-a967-cdb1b6c57cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:52.967394 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16c830-3937-4c57-a967-cdb1b6c57cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:52.967663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16c830-3937-4c57-a967-cdb1b6c57cb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:52.977896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c16c830-3937-4c57-a967-cdb1b6c57cb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:24:52.982962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-8c16c830-3937-4c57-a967-cdb1b6c57cb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:24:52.982962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:24:52.982962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:24:52.982962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:24:52.982962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:24:52.982962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:24:52.982962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:24:52.982962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:24:52.982962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:24:52.982962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:24:52.982962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:24:52.982962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:24:52.982962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:24:52.982962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:24:52.982962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:24:52.982962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:24:52.984022 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:24:52.984022 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:24:52.984022 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:24:52.984022 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:24:52.984022 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:24:52.984022 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:24:52.984022 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:24:52.984022 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:24:52.984022 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:24:52.984022 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:24:52.984022 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:24:52.984022 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:24:52.984022 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:24:52.984022 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:24:52.984022 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:24:52.984022 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:24:52.984985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:24:52.984985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:24:52.984985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:24:52.984985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:24:52.984985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:24:52.984985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:24:52.984985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:24:52.984985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:24:52.984985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:24:52.984985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:24:52.984985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:24:52.984985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:24:52.984985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 599/1195] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:24:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:24:52.990330 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc29a5-0890-4023-b4d5-80b66f7ff89b None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:53.088790 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc29a5-0890-4023-b4d5-80b66f7ff89b None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:53.121484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1329a8-9910-4614-af24-b372f5a2cd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:53.122199 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1329a8-9910-4614-af24-b372f5a2cd62 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:53.122476 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1329a8-9910-4614-af24-b372f5a2cd62 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:53.122745 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1329a8-9910-4614-af24-b372f5a2cd62 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:53.123447 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1329a8-9910-4614-af24-b372f5a2cd62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:53.123743 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1329a8-9910-4614-af24-b372f5a2cd62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:53.125133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1329a8-9910-4614-af24-b372f5a2cd62 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:53.133001 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 597/1196] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:24:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:24:53.139250 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16c830-3937-4c57-a967-cdb1b6c57cb7 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:53.174263 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:53.245784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16c830-3937-4c57-a967-cdb1b6c57cb7 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:53.279231 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32b99e-820b-4e82-aa2c-f0291ff98693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:53.279869 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32b99e-820b-4e82-aa2c-f0291ff98693 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:53.280150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32b99e-820b-4e82-aa2c-f0291ff98693 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:53.280427 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32b99e-820b-4e82-aa2c-f0291ff98693 None admin] PATH_INFO: `/v3/users/d7e07cb9e74d4bed8c3e9671c76dbc0c` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:53.281586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32b99e-820b-4e82-aa2c-f0291ff98693 None admin] RBAC: Authorizing `identity:get_user(user_id=d7e07cb9e74d4bed8c3e9671c76dbc0c)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:53.283018 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32b99e-820b-4e82-aa2c-f0291ff98693 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:53.284615 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 600/1197] 10.222.0.74 () {56 vars in 1177 bytes} [Wed Jul 27 16:24:53 2022] GET /identity/v3/users/d7e07cb9e74d4bed8c3e9671c76dbc0c => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:53.290414 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1329a8-9910-4614-af24-b372f5a2cd62 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:53.331580 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:53.387554 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1329a8-9910-4614-af24-b372f5a2cd62 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:53.420160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1bc98-3dcf-44a3-ae71-a2414d5e0340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:53.420760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1bc98-3dcf-44a3-ae71-a2414d5e0340 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:53.421068 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1bc98-3dcf-44a3-ae71-a2414d5e0340 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:53.421343 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1bc98-3dcf-44a3-ae71-a2414d5e0340 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:53.485745 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1bc98-3dcf-44a3-ae71-a2414d5e0340 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:53.487193 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1bc98-3dcf-44a3-ae71-a2414d5e0340 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:53.526301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:53.582550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 598/1198] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:24:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 16:24:53.588954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32b99e-820b-4e82-aa2c-f0291ff98693 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:53.688532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:24:53.691627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32b99e-820b-4e82-aa2c-f0291ff98693 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:53.725581 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941b722-d416-4c6c-b372-d701f1bb7691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:53.726206 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941b722-d416-4c6c-b372-d701f1bb7691 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:53.726485 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941b722-d416-4c6c-b372-d701f1bb7691 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:53.726758 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941b722-d416-4c6c-b372-d701f1bb7691 None admin] PATH_INFO: `/v3/projects/ec38d216b5544ec2857365422e6f7408` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:53.727887 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0941b722-d416-4c6c-b372-d701f1bb7691 None admin] RBAC: Authorizing `identity:get_project(project_id=ec38d216b5544ec2857365422e6f7408)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:53.729364 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0941b722-d416-4c6c-b372-d701f1bb7691 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:53.730999 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 601/1199] 10.222.0.74 () {56 vars in 1186 bytes} [Wed Jul 27 16:24:53 2022] GET /identity/v3/projects/ec38d216b5544ec2857365422e6f7408 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:24:53.737035 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1bc98-3dcf-44a3-ae71-a2414d5e0340 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:24:53.831975 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1bc98-3dcf-44a3-ae71-a2414d5e0340 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:24:53.864428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ca7f0-b6e8-422f-8c82-4923936ed765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:24:53.865051 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ca7f0-b6e8-422f-8c82-4923936ed765 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:24:53.865333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ca7f0-b6e8-422f-8c82-4923936ed765 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:24:53.865605 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ca7f0-b6e8-422f-8c82-4923936ed765 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:24:53.866374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ca7f0-b6e8-422f-8c82-4923936ed765 None admin] RBAC: Adding query filter params (role.id=ca37e43082fa40c6a88b12680a0ca94b, scope.project.id=ec38d216b5544ec2857365422e6f7408, user.id=d7e07cb9e74d4bed8c3e9671c76dbc0c) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:24:53.866678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ca7f0-b6e8-422f-8c82-4923936ed765 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:24:53.868191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ca7f0-b6e8-422f-8c82-4923936ed765 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:24:53.873775 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 599/1200] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:24:53 2022] GET /identity/v3/role_assignments?user.id=d7e07cb9e74d4bed8c3e9671c76dbc0c&scope.project.id=ec38d216b5544ec2857365422e6f7408&role.id=ca37e43082fa40c6a88b12680a0ca94b => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:25:03.721562 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=16 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:25:03.871277 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=98 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:25:48.584213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a925c0-f4b1-4ea2-a7e5-80846c443f2e None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:48.584865 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a925c0-f4b1-4ea2-a7e5-80846c443f2e None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:48.584865 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a925c0-f4b1-4ea2-a7e5-80846c443f2e None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:48.586510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 602/1201] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:25:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:25:48.592802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de625cd-3b83-4d7c-814d-135c633dabfe None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:48.593292 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de625cd-3b83-4d7c-814d-135c633dabfe None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:48.593430 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de625cd-3b83-4d7c-814d-135c633dabfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:48.612877 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-6de625cd-3b83-4d7c-814d-135c633dabfe None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:25:48.614269 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6de625cd-3b83-4d7c-814d-135c633dabfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:25:48.709735 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 600/1202] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:25:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:25:48.738484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a925c0-f4b1-4ea2-a7e5-80846c443f2e None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:48.883706 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a925c0-f4b1-4ea2-a7e5-80846c443f2e None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:48.930495 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112d534-69f6-47c2-b006-9eba73b24d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:48.931213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112d534-69f6-47c2-b006-9eba73b24d86 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:48.931494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112d534-69f6-47c2-b006-9eba73b24d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:48.931770 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112d534-69f6-47c2-b006-9eba73b24d86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:48.945047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112d534-69f6-47c2-b006-9eba73b24d86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:48.946614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112d534-69f6-47c2-b006-9eba73b24d86 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:48.964627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-7112d534-69f6-47c2-b006-9eba73b24d86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:25:48.964627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:25:48.964627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:25:48.964627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:25:48.964627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:25:48.964627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:25:48.964627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:25:48.964627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:25:48.964627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:25:48.964627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:25:48.964627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:25:48.964627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:25:48.964627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:25:48.964627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:25:48.964627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:25:48.965705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:25:48.965705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:25:48.965705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:25:48.965705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:25:48.965705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:25:48.965705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:25:48.965705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:25:48.965705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:25:48.965705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:25:48.965705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:25:48.965705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:25:48.965705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:25:48.965705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:25:48.965705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:25:48.965705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:25:48.965705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:25:48.965705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:25:48.966661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:25:48.966661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:25:48.966661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:25:48.966661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:25:48.966661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 603/1203] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:25:48.973095 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de625cd-3b83-4d7c-814d-135c633dabfe None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:49.120000 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de625cd-3b83-4d7c-814d-135c633dabfe None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:49.151749 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:25:49.160256 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd183108-8a15-429a-977b-0648a1d55a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:49.160925 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd183108-8a15-429a-977b-0648a1d55a24 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:49.161213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd183108-8a15-429a-977b-0648a1d55a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:49.161477 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd183108-8a15-429a-977b-0648a1d55a24 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:49.165020 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd183108-8a15-429a-977b-0648a1d55a24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:25:49.165020 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd183108-8a15-429a-977b-0648a1d55a24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:49.165020 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd183108-8a15-429a-977b-0648a1d55a24 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:49.180378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 601/1204] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:25:49.188005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112d534-69f6-47c2-b006-9eba73b24d86 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:49.273925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:25:49.376136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112d534-69f6-47c2-b006-9eba73b24d86 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:49.426558 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3dc5b-dffa-46cd-8e59-7b4073e4b7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:49.427122 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3dc5b-dffa-46cd-8e59-7b4073e4b7ca None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:49.427398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3dc5b-dffa-46cd-8e59-7b4073e4b7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:49.427674 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3dc5b-dffa-46cd-8e59-7b4073e4b7ca None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:49.428375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d3dc5b-dffa-46cd-8e59-7b4073e4b7ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:49.429918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d3dc5b-dffa-46cd-8e59-7b4073e4b7ca None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:49.458583 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 604/1205] 10.222.0.74 () {60 vars in 1132 bytes} [Wed Jul 27 16:25:49 2022] POST /identity/v3/users => generated 251 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:25:51.710745 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781bc19c-47b7-4800-8d38-ac9e5a5d1972 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:51.711346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781bc19c-47b7-4800-8d38-ac9e5a5d1972 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:51.711346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781bc19c-47b7-4800-8d38-ac9e5a5d1972 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:51.713369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 602/1206] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:25:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:25:51.722187 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3dca9-f96a-4483-be07-a6378f47a628 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:51.722187 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3dca9-f96a-4483-be07-a6378f47a628 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:51.722187 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3dca9-f96a-4483-be07-a6378f47a628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:51.739619 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-deb3dca9-f96a-4483-be07-a6378f47a628 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:25:51.740992 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-deb3dca9-f96a-4483-be07-a6378f47a628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:25:51.824923 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 605/1207] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:25:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:25:51.845194 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781bc19c-47b7-4800-8d38-ac9e5a5d1972 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:51.988484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781bc19c-47b7-4800-8d38-ac9e5a5d1972 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:52.034597 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88bec8-c815-4606-bb51-bca27634fc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:52.034597 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88bec8-c815-4606-bb51-bca27634fc2f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:52.035085 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88bec8-c815-4606-bb51-bca27634fc2f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:52.035334 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88bec8-c815-4606-bb51-bca27634fc2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:52.057026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88bec8-c815-4606-bb51-bca27634fc2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:25:52.064079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-dd88bec8-c815-4606-bb51-bca27634fc2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:25:52.064079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:25:52.064079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:25:52.064079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:25:52.064079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:25:52.064079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:25:52.064079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:25:52.064079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:25:52.064079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:25:52.064079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:25:52.064079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:25:52.064079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:25:52.064079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:25:52.064079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:25:52.064079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:25:52.064079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:25:52.065342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:25:52.065342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:25:52.065342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:25:52.065342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:25:52.065342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:25:52.065342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:25:52.065342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:25:52.065342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:25:52.065342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:25:52.065342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:25:52.065342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:25:52.065342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:25:52.065342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:25:52.065342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:25:52.065342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:25:52.065342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:25:52.066252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:25:52.066252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:25:52.066252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:25:52.066252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:25:52.066252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:25:52.066252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:25:52.066252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:25:52.066252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:25:52.066252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:25:52.066252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:25:52.066252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:25:52.066252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:25:52.066252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 603/1208] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:25:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:25:52.072032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3dca9-f96a-4483-be07-a6378f47a628 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:52.119412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:25:52.204905 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3dca9-f96a-4483-be07-a6378f47a628 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:52.249767 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e8d49-1fcc-4463-8ea5-444fb5cd5abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:52.250348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e8d49-1fcc-4463-8ea5-444fb5cd5abc None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:52.250659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e8d49-1fcc-4463-8ea5-444fb5cd5abc None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:52.250968 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e8d49-1fcc-4463-8ea5-444fb5cd5abc None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:52.251751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e8d49-1fcc-4463-8ea5-444fb5cd5abc None admin] RBAC: Adding query filter params (name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:25:52.252191 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e8d49-1fcc-4463-8ea5-444fb5cd5abc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:52.254002 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e8d49-1fcc-4463-8ea5-444fb5cd5abc None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:52.269746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 606/1209] 10.222.0.74 () {56 vars in 1103 bytes} [Wed Jul 27 16:25:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:25:52.272051 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88bec8-c815-4606-bb51-bca27634fc2f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:52.336509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:25:52.396880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88bec8-c815-4606-bb51-bca27634fc2f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:52.437144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23423b74-8f3b-4642-8a63-92199691ea5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:52.438246 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23423b74-8f3b-4642-8a63-92199691ea5b None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:52.438619 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23423b74-8f3b-4642-8a63-92199691ea5b None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:52.438958 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23423b74-8f3b-4642-8a63-92199691ea5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:52.456511 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23423b74-8f3b-4642-8a63-92199691ea5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:52.458481 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23423b74-8f3b-4642-8a63-92199691ea5b None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:52.479075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-23423b74-8f3b-4642-8a63-92199691ea5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:25:52.479075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:25:52.479075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:25:52.479075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:25:52.479075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:25:52.479075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:25:52.479075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:25:52.479075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:25:52.479075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:25:52.479075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:25:52.479075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:25:52.479075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:25:52.479075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:25:52.479075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:25:52.479075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:25:52.480405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:25:52.480405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:25:52.480405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:25:52.480405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:25:52.480405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:25:52.480405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:25:52.480405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:25:52.480405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:25:52.480405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:25:52.480405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:25:52.480405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:25:52.480405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:25:52.480405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:25:52.480405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:25:52.480405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:25:52.480405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:25:52.480405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:25:52.481429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:25:52.481429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:25:52.481429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:25:52.481429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:25:52.481429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 604/1210] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:25:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:25:52.486537 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e8d49-1fcc-4463-8ea5-444fb5cd5abc None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:52.626599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:25:52.632277 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e8d49-1fcc-4463-8ea5-444fb5cd5abc None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:52.680737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de807c8-85ba-4c13-8d16-71f48716bfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:52.681595 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de807c8-85ba-4c13-8d16-71f48716bfb5 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:52.681916 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de807c8-85ba-4c13-8d16-71f48716bfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:52.682399 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de807c8-85ba-4c13-8d16-71f48716bfb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:52.683584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de807c8-85ba-4c13-8d16-71f48716bfb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:25:52.683849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de807c8-85ba-4c13-8d16-71f48716bfb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:52.686089 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de807c8-85ba-4c13-8d16-71f48716bfb5 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:52.706147 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 607/1211] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:25:52.709525 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23423b74-8f3b-4642-8a63-92199691ea5b None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:52.835647 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23423b74-8f3b-4642-8a63-92199691ea5b None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:52.837495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:25:52.888604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901cb33b-06d4-4c11-91f2-202e88d59daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:52.889212 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901cb33b-06d4-4c11-91f2-202e88d59daa None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:52.889496 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901cb33b-06d4-4c11-91f2-202e88d59daa None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:52.889818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901cb33b-06d4-4c11-91f2-202e88d59daa None admin] PATH_INFO: `/v3/users/placement` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:52.899975 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901cb33b-06d4-4c11-91f2-202e88d59daa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:52.902070 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901cb33b-06d4-4c11-91f2-202e88d59daa None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:52.920220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-901cb33b-06d4-4c11-91f2-202e88d59daa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 16:25:52.920220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:25:52.920220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:25:52.920220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:25:52.920220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:25:52.920220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:25:52.920220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:25:52.920220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:25:52.920220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:25:52.920220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:25:52.920220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:25:52.920220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:25:52.920220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:25:52.920220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:25:52.920220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:25:52.920220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:25:52.922191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:25:52.922191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:25:52.922191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:25:52.922191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:25:52.922191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:25:52.922191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:25:52.922191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:25:52.922191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:25:52.922191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:25:52.922191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:25:52.922191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:25:52.922191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:25:52.922191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:25:52.922191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:25:52.922191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:25:52.922191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:25:52.922191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:25:52.922191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:25:52.924517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:25:52.924517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:25:52.924517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:25:52.924517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:25:52.924517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:25:52.924517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:25:52.924517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:25:52.924517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:25:52.924517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 16:25:52.924517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:25:52.924517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 605/1212] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:25:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:25:52.928260 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de807c8-85ba-4c13-8d16-71f48716bfb5 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:53.057230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de807c8-85ba-4c13-8d16-71f48716bfb5 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:53.104644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffd80e-5f45-4ec5-800f-11c293bf44bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:53.105629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffd80e-5f45-4ec5-800f-11c293bf44bc None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:53.106759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffd80e-5f45-4ec5-800f-11c293bf44bc None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:53.106759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffd80e-5f45-4ec5-800f-11c293bf44bc None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:53.107767 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ffd80e-5f45-4ec5-800f-11c293bf44bc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:25:53.108241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ffd80e-5f45-4ec5-800f-11c293bf44bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:53.110570 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ffd80e-5f45-4ec5-800f-11c293bf44bc None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:53.135861 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 608/1213] 10.222.0.74 () {56 vars in 1143 bytes} [Wed Jul 27 16:25:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:25:53.141965 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901cb33b-06d4-4c11-91f2-202e88d59daa None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:53.272521 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901cb33b-06d4-4c11-91f2-202e88d59daa None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:53.313863 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c087f82-c98a-4c13-a7d6-9ba632f629c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:53.314664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c087f82-c98a-4c13-a7d6-9ba632f629c5 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:53.314986 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c087f82-c98a-4c13-a7d6-9ba632f629c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:53.315268 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c087f82-c98a-4c13-a7d6-9ba632f629c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:53.353381 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:25:53.408205 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c087f82-c98a-4c13-a7d6-9ba632f629c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:53.409899 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c087f82-c98a-4c13-a7d6-9ba632f629c5 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:53.533537 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 606/1214] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:25:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 16:25:53.540556 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffd80e-5f45-4ec5-800f-11c293bf44bc None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:53.565065 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:25:53.657196 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffd80e-5f45-4ec5-800f-11c293bf44bc None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:53.696663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdeaec4-5230-4589-8557-01989db75dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:53.697374 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdeaec4-5230-4589-8557-01989db75dda None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:53.697727 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdeaec4-5230-4589-8557-01989db75dda None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:53.698075 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdeaec4-5230-4589-8557-01989db75dda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:53.710032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdeaec4-5230-4589-8557-01989db75dda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:53.712176 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdeaec4-5230-4589-8557-01989db75dda None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:53.728471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-7cdeaec4-5230-4589-8557-01989db75dda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:25:53.728471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:25:53.728471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:25:53.728471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:25:53.728471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:25:53.728471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:25:53.728471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:25:53.728471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:25:53.728471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:25:53.728471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:25:53.728471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:25:53.728471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:25:53.728471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:25:53.728471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:25:53.728471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:25:53.730585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:25:53.730585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:25:53.730585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:25:53.730585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:25:53.730585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:25:53.730585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:25:53.730585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:25:53.730585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:25:53.730585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:25:53.730585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:25:53.730585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:25:53.730585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:25:53.730585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:25:53.730585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:25:53.730585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:25:53.730585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:25:53.730585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:25:53.731712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:25:53.731712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:25:53.731712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:25:53.731712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:25:53.735158 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 609/1215] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:25:53.736604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c087f82-c98a-4c13-a7d6-9ba632f629c5 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:53.806782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:25:53.851121 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c087f82-c98a-4c13-a7d6-9ba632f629c5 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:53.893527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685a2b9-e6af-49b7-bb8e-8fc769abc7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:53.895218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685a2b9-e6af-49b7-bb8e-8fc769abc7d8 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:53.895218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685a2b9-e6af-49b7-bb8e-8fc769abc7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:53.895218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685a2b9-e6af-49b7-bb8e-8fc769abc7d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:53.895778 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685a2b9-e6af-49b7-bb8e-8fc769abc7d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:25:53.896095 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685a2b9-e6af-49b7-bb8e-8fc769abc7d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:53.898377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685a2b9-e6af-49b7-bb8e-8fc769abc7d8 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:53.914336 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 607/1216] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:25:53.920719 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdeaec4-5230-4589-8557-01989db75dda None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:54.005307 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:25:54.042152 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdeaec4-5230-4589-8557-01989db75dda None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:54.080362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ec518-4e93-4885-b1a6-9f35257f2996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:54.081126 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ec518-4e93-4885-b1a6-9f35257f2996 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:54.081574 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ec518-4e93-4885-b1a6-9f35257f2996 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:54.081965 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ec518-4e93-4885-b1a6-9f35257f2996 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:54.091475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ec518-4e93-4885-b1a6-9f35257f2996 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:54.093642 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ec518-4e93-4885-b1a6-9f35257f2996 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:54.107081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-a61ec518-4e93-4885-b1a6-9f35257f2996 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:25:54.107081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:25:54.107081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:25:54.107081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:25:54.107081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:25:54.107081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:25:54.107081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:25:54.107081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:25:54.107081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:25:54.107081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:25:54.107081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:25:54.107081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:25:54.107081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:25:54.107081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:25:54.107081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:25:54.108222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:25:54.108222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:25:54.108222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:25:54.108222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:25:54.108222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:25:54.108222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:25:54.108222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:25:54.108222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:25:54.108222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:25:54.108222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:25:54.108222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:25:54.108222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:25:54.108222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:25:54.108222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:25:54.108222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:25:54.108222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:25:54.108222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:25:54.109344 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:25:54.109344 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:25:54.109344 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:25:54.109344 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:25:54.109344 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:25:54.109344 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:25:54.109344 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:25:54.109344 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:25:54.116773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 610/1217] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:25:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:25:54.116960 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685a2b9-e6af-49b7-bb8e-8fc769abc7d8 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:54.233917 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685a2b9-e6af-49b7-bb8e-8fc769abc7d8 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:54.257879 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:25:54.273521 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69e19b-01ce-4941-836e-0ab04e3a9d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:54.274225 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69e19b-01ce-4941-836e-0ab04e3a9d07 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:54.274577 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69e19b-01ce-4941-836e-0ab04e3a9d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:54.274944 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69e19b-01ce-4941-836e-0ab04e3a9d07 None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:54.275762 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de69e19b-01ce-4941-836e-0ab04e3a9d07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:25:54.276136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de69e19b-01ce-4941-836e-0ab04e3a9d07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:54.277669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de69e19b-01ce-4941-836e-0ab04e3a9d07 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:54.296929 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 608/1218] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:25:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:25:54.300983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ec518-4e93-4885-b1a6-9f35257f2996 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:54.416033 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ec518-4e93-4885-b1a6-9f35257f2996 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:54.448078 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:25:54.456417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9202ba-21d9-49cf-8bc0-d32bb7c601a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:54.457305 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9202ba-21d9-49cf-8bc0-d32bb7c601a9 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:54.457777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9202ba-21d9-49cf-8bc0-d32bb7c601a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:54.458233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9202ba-21d9-49cf-8bc0-d32bb7c601a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:54.459225 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9202ba-21d9-49cf-8bc0-d32bb7c601a9 None admin] RBAC: Adding query filter params (role.id=2aa0cdfed849496db8f3692c9569d7ee, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=91f8e0f82a8541aead6cd8cc45f4a86b) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:25:54.459722 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9202ba-21d9-49cf-8bc0-d32bb7c601a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:54.461361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9202ba-21d9-49cf-8bc0-d32bb7c601a9 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:54.467257 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 611/1219] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:25:54 2022] GET /identity/v3/role_assignments?user.id=91f8e0f82a8541aead6cd8cc45f4a86b&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=2aa0cdfed849496db8f3692c9569d7ee => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:25:56.463384 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f07721-341f-4d11-bbb4-fbd416407713 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:56.463923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f07721-341f-4d11-bbb4-fbd416407713 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:56.464047 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f07721-341f-4d11-bbb4-fbd416407713 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:56.465361 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 609/1220] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:25:56.470421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c579afa-93ca-45c2-9ec2-0d554cdf0aaf None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:56.470864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c579afa-93ca-45c2-9ec2-0d554cdf0aaf None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:56.470967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c579afa-93ca-45c2-9ec2-0d554cdf0aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:56.491085 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-5c579afa-93ca-45c2-9ec2-0d554cdf0aaf None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:25:56.492909 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-5c579afa-93ca-45c2-9ec2-0d554cdf0aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:25:56.567041 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 612/1221] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:25:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:25:56.590450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f07721-341f-4d11-bbb4-fbd416407713 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:56.706309 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f07721-341f-4d11-bbb4-fbd416407713 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:56.740938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf98501-bfeb-42fc-aea8-d7258d73df14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:56.742123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf98501-bfeb-42fc-aea8-d7258d73df14 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:56.742401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf98501-bfeb-42fc-aea8-d7258d73df14 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:56.742671 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf98501-bfeb-42fc-aea8-d7258d73df14 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:56.752663 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf98501-bfeb-42fc-aea8-d7258d73df14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:25:56.757553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-7cf98501-bfeb-42fc-aea8-d7258d73df14 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:25:56.757553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:25:56.757553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:25:56.757553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:25:56.757553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:25:56.757553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:25:56.757553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:25:56.757553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:25:56.757553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:25:56.757553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:25:56.757553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:25:56.757553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:25:56.757553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:25:56.757553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:25:56.757553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:25:56.757553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:25:56.758527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:25:56.758527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:25:56.758527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:25:56.758527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:25:56.758527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:25:56.758527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:25:56.758527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:25:56.758527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:25:56.758527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:25:56.758527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:25:56.758527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:25:56.758527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:25:56.758527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:25:56.758527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:25:56.758527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:25:56.758527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:25:56.759405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:25:56.759405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:25:56.759405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:25:56.759405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:25:56.759405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:25:56.759405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:25:56.759405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:25:56.759405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:25:56.759405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:25:56.759405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:25:56.759405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:25:56.759405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:25:56.759405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 610/1222] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:25:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:25:56.765020 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c579afa-93ca-45c2-9ec2-0d554cdf0aaf None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:56.918679 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c579afa-93ca-45c2-9ec2-0d554cdf0aaf None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:56.953353 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:25:56.966893 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89a877-91a0-4f48-aa40-81be4a3122fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:56.967680 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89a877-91a0-4f48-aa40-81be4a3122fd None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:56.968151 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89a877-91a0-4f48-aa40-81be4a3122fd None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:56.968613 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89a877-91a0-4f48-aa40-81be4a3122fd None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:56.969898 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89a877-91a0-4f48-aa40-81be4a3122fd None admin] RBAC: Adding query filter params (name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:25:56.970414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89a877-91a0-4f48-aa40-81be4a3122fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:56.972904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89a877-91a0-4f48-aa40-81be4a3122fd None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:56.986881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 613/1223] 10.222.0.74 () {56 vars in 1103 bytes} [Wed Jul 27 16:25:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:25:56.993405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf98501-bfeb-42fc-aea8-d7258d73df14 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:57.039894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:25:57.101697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf98501-bfeb-42fc-aea8-d7258d73df14 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:57.138180 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc242e3b-90e3-429d-9fae-5b5da12f17f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:57.139133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc242e3b-90e3-429d-9fae-5b5da12f17f7 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:57.139420 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc242e3b-90e3-429d-9fae-5b5da12f17f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:57.139695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc242e3b-90e3-429d-9fae-5b5da12f17f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:57.153719 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc242e3b-90e3-429d-9fae-5b5da12f17f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:57.155417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc242e3b-90e3-429d-9fae-5b5da12f17f7 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:57.174210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-fc242e3b-90e3-429d-9fae-5b5da12f17f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:25:57.174210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:25:57.174210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:25:57.174210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:25:57.174210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:25:57.174210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:25:57.174210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:25:57.174210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:25:57.174210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:25:57.174210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:25:57.174210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:25:57.174210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:25:57.174210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:25:57.174210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:25:57.174210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:25:57.175165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:25:57.175165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:25:57.175165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:25:57.175165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:25:57.175165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:25:57.175165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:25:57.175165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:25:57.175165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:25:57.175165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:25:57.175165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:25:57.175165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:25:57.175165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:25:57.175165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:25:57.175165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:25:57.175165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:25:57.175165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:25:57.175165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:25:57.176075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:25:57.176075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:25:57.176075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:25:57.176075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:25:57.176075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 611/1224] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:25:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:25:57.183200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89a877-91a0-4f48-aa40-81be4a3122fd None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:57.296842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89a877-91a0-4f48-aa40-81be4a3122fd None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:57.331649 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d439b-9f6e-4548-84e8-d0963bae2dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:57.332267 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d439b-9f6e-4548-84e8-d0963bae2dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:57.332586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d439b-9f6e-4548-84e8-d0963bae2dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:57.332906 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d439b-9f6e-4548-84e8-d0963bae2dc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:57.333653 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d439b-9f6e-4548-84e8-d0963bae2dc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:25:57.333998 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d439b-9f6e-4548-84e8-d0963bae2dc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:57.336226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d439b-9f6e-4548-84e8-d0963bae2dc5 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:57.351195 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 614/1225] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:25:57.357169 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc242e3b-90e3-429d-9fae-5b5da12f17f7 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:57.460023 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:25:57.469412 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc242e3b-90e3-429d-9fae-5b5da12f17f7 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:57.510411 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b3bad-cb5a-4b3f-b9ae-4b584b0a59c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:57.511054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b3bad-cb5a-4b3f-b9ae-4b584b0a59c4 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:57.511340 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b3bad-cb5a-4b3f-b9ae-4b584b0a59c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:57.511620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b3bad-cb5a-4b3f-b9ae-4b584b0a59c4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:57.520081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b3bad-cb5a-4b3f-b9ae-4b584b0a59c4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:57.521822 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b3bad-cb5a-4b3f-b9ae-4b584b0a59c4 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:57.533812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-b10b3bad-cb5a-4b3f-b9ae-4b584b0a59c4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 16:25:57.533812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:25:57.533812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:25:57.533812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:25:57.533812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:25:57.533812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:25:57.533812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:25:57.533812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:25:57.533812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:25:57.533812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:25:57.533812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:25:57.533812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:25:57.533812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:25:57.533812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:25:57.533812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:25:57.533812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:25:57.535026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:25:57.535026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:25:57.535026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:25:57.535026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:25:57.535026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:25:57.535026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:25:57.535026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:25:57.535026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:25:57.535026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:25:57.535026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:25:57.535026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:25:57.535026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:25:57.535026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:25:57.535026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:25:57.535026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:25:57.535026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:25:57.535026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:25:57.535026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:25:57.536053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:25:57.536053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:25:57.536053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:25:57.536053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:25:57.536053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:25:57.536053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:25:57.536053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:25:57.536053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:25:57.536053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 16:25:57.536053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:25:57.536053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 612/1226] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:25:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:25:57.540730 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d439b-9f6e-4548-84e8-d0963bae2dc5 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:57.592451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:25:57.656685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d439b-9f6e-4548-84e8-d0963bae2dc5 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:57.692778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11d90d-523b-4b38-a215-4bfb499883be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:57.693431 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11d90d-523b-4b38-a215-4bfb499883be None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:57.693714 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11d90d-523b-4b38-a215-4bfb499883be None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:57.693997 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11d90d-523b-4b38-a215-4bfb499883be None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:57.695349 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c11d90d-523b-4b38-a215-4bfb499883be None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:25:57.695675 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c11d90d-523b-4b38-a215-4bfb499883be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:57.697183 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c11d90d-523b-4b38-a215-4bfb499883be None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:57.718684 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 615/1227] 10.222.0.74 () {56 vars in 1143 bytes} [Wed Jul 27 16:25:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:25:57.725039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b3bad-cb5a-4b3f-b9ae-4b584b0a59c4 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:57.846924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b3bad-cb5a-4b3f-b9ae-4b584b0a59c4 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:57.884084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219e9d4-ef14-4aec-a73c-0cdb02d7f6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:57.884800 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219e9d4-ef14-4aec-a73c-0cdb02d7f6db None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:57.885113 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219e9d4-ef14-4aec-a73c-0cdb02d7f6db None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:57.885389 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219e9d4-ef14-4aec-a73c-0cdb02d7f6db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:57.896619 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219e9d4-ef14-4aec-a73c-0cdb02d7f6db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:57.897985 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219e9d4-ef14-4aec-a73c-0cdb02d7f6db None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:57.912641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-0219e9d4-ef14-4aec-a73c-0cdb02d7f6db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:25:57.912641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:25:57.912641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:25:57.912641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:25:57.912641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:25:57.912641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:25:57.912641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:25:57.912641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:25:57.912641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:25:57.912641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:25:57.912641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:25:57.912641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:25:57.912641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:25:57.912641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:25:57.912641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:25:57.913678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:25:57.913678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:25:57.913678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:25:57.913678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:25:57.913678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:25:57.913678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:25:57.913678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:25:57.913678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:25:57.913678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:25:57.913678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:25:57.913678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:25:57.913678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:25:57.913678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:25:57.913678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:25:57.913678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:25:57.913678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:25:57.913678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:25:57.914611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:25:57.914611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:25:57.914611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:25:57.914611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:25:57.914611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 613/1228] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:25:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:25:57.919695 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11d90d-523b-4b38-a215-4bfb499883be None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:58.029895 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11d90d-523b-4b38-a215-4bfb499883be None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:58.032404 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:25:58.073227 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f1cba-87c3-4730-90f4-b7baf4a30a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:58.073874 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f1cba-87c3-4730-90f4-b7baf4a30a21 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:58.074162 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f1cba-87c3-4730-90f4-b7baf4a30a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:58.074581 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f1cba-87c3-4730-90f4-b7baf4a30a21 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:58.075274 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f1cba-87c3-4730-90f4-b7baf4a30a21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:25:58.075577 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f1cba-87c3-4730-90f4-b7baf4a30a21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:58.077406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f1cba-87c3-4730-90f4-b7baf4a30a21 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:58.090741 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 616/1229] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:25:58.098734 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219e9d4-ef14-4aec-a73c-0cdb02d7f6db None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:58.111764 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:25:58.216919 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219e9d4-ef14-4aec-a73c-0cdb02d7f6db None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:58.253377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a8a54-c518-4dec-8acd-a058cdac739d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:58.254015 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a8a54-c518-4dec-8acd-a058cdac739d None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:58.254293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a8a54-c518-4dec-8acd-a058cdac739d None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:58.254564 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a8a54-c518-4dec-8acd-a058cdac739d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:58.261522 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a8a54-c518-4dec-8acd-a058cdac739d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:58.263043 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a8a54-c518-4dec-8acd-a058cdac739d None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:58.275898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-2c9a8a54-c518-4dec-8acd-a058cdac739d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:25:58.275898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:25:58.275898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:25:58.275898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:25:58.275898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:25:58.275898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:25:58.275898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:25:58.275898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:25:58.275898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:25:58.275898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:25:58.275898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:25:58.275898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:25:58.275898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:25:58.275898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:25:58.275898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:25:58.276848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:25:58.276848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:25:58.276848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:25:58.276848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:25:58.276848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:25:58.276848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:25:58.276848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:25:58.276848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:25:58.276848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:25:58.276848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:25:58.276848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:25:58.276848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:25:58.276848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:25:58.276848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:25:58.276848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:25:58.276848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:25:58.276848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:25:58.278137 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:25:58.278137 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:25:58.278137 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:25:58.278137 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:25:58.278137 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:25:58.278137 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:25:58.278137 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:25:58.278137 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:25:58.278137 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 614/1230] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:25:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:25:58.283435 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f1cba-87c3-4730-90f4-b7baf4a30a21 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:58.404001 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f1cba-87c3-4730-90f4-b7baf4a30a21 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:58.445739 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75aa5a-30dc-4d22-aa40-8ee618284cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:58.446484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75aa5a-30dc-4d22-aa40-8ee618284cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:58.446836 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75aa5a-30dc-4d22-aa40-8ee618284cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:58.447148 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75aa5a-30dc-4d22-aa40-8ee618284cd4 None admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:58.447956 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75aa5a-30dc-4d22-aa40-8ee618284cd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:25:58.448298 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75aa5a-30dc-4d22-aa40-8ee618284cd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:58.449915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75aa5a-30dc-4d22-aa40-8ee618284cd4 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:58.464323 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 617/1231] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:25:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:25:58.471000 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a8a54-c518-4dec-8acd-a058cdac739d None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:25:58.544370 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:25:58.582581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a8a54-c518-4dec-8acd-a058cdac739d None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:25:58.618755 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369f632-d928-4928-8e6b-74c914d83794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:25:58.619540 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369f632-d928-4928-8e6b-74c914d83794 None admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:25:58.619885 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369f632-d928-4928-8e6b-74c914d83794 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:25:58.620226 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369f632-d928-4928-8e6b-74c914d83794 None admin] PATH_INFO: `/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/91f8e0f82a8541aead6cd8cc45f4a86b/roles/2aa0cdfed849496db8f3692c9569d7ee` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:25:58.637443 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369f632-d928-4928-8e6b-74c914d83794 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d6bcdd09cf24a5c8003951db37020cf, user_id=91f8e0f82a8541aead6cd8cc45f4a86b, role_id=2aa0cdfed849496db8f3692c9569d7ee)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:25:58.639139 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369f632-d928-4928-8e6b-74c914d83794 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:25:58.652395 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 615/1232] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:25:58 2022] PUT /identity/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/91f8e0f82a8541aead6cd8cc45f4a86b/roles/2aa0cdfed849496db8f3692c9569d7ee => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:26:00.765747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e7193-66e1-4d3a-a88c-7ca6f6ec36b1 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:00.766300 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e7193-66e1-4d3a-a88c-7ca6f6ec36b1 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:00.766300 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e7193-66e1-4d3a-a88c-7ca6f6ec36b1 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:00.767894 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 618/1233] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:26:00.773540 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6d141-4bdd-4e1e-9fb7-02a9e0a6020f None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:00.774000 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6d141-4bdd-4e1e-9fb7-02a9e0a6020f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:00.774190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6d141-4bdd-4e1e-9fb7-02a9e0a6020f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:00.797253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-3ce6d141-4bdd-4e1e-9fb7-02a9e0a6020f None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:26:00.799246 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-3ce6d141-4bdd-4e1e-9fb7-02a9e0a6020f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:26:00.872398 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 616/1234] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:26:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:26:00.895985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e7193-66e1-4d3a-a88c-7ca6f6ec36b1 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:00.930409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:01.023360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e7193-66e1-4d3a-a88c-7ca6f6ec36b1 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:01.059481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b306663-9340-49bb-964d-ea16a4a2e40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:01.060127 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b306663-9340-49bb-964d-ea16a4a2e40a None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:01.060410 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b306663-9340-49bb-964d-ea16a4a2e40a None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:01.060688 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b306663-9340-49bb-964d-ea16a4a2e40a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:01.071043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b306663-9340-49bb-964d-ea16a4a2e40a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:26:01.076457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-3b306663-9340-49bb-964d-ea16a4a2e40a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:26:01.076457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:01.076457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:01.076457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:01.076457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:01.076457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:01.076457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:01.076457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:01.076457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:01.076457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:01.076457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:01.076457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:01.076457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:26:01.076457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:26:01.076457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:26:01.076457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:26:01.078207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:26:01.078207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:26:01.078207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:26:01.078207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:26:01.078207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:01.078207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:01.078207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:01.078207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:01.078207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:01.078207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:01.078207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:01.078207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:01.078207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:01.078207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:01.078207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:01.078207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:01.079071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:01.079071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:01.079071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:01.079071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:01.079071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:26:01.079071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:26:01.079071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:26:01.079071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:26:01.079071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:26:01.079071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:26:01.079071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 16:26:01.079071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:26:01.079071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 619/1235] 10.222.0.74 () {56 vars in 1102 bytes} [Wed Jul 27 16:26:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:26:01.084292 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6d141-4bdd-4e1e-9fb7-02a9e0a6020f None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:01.132984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:01.186997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6d141-4bdd-4e1e-9fb7-02a9e0a6020f None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:01.222043 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8913c-a920-4e24-b597-d18dc0e06739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:01.222909 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8913c-a920-4e24-b597-d18dc0e06739 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:01.223295 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8913c-a920-4e24-b597-d18dc0e06739 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:01.223643 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8913c-a920-4e24-b597-d18dc0e06739 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:01.224434 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8913c-a920-4e24-b597-d18dc0e06739 None admin] RBAC: Adding query filter params (name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:01.224813 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8913c-a920-4e24-b597-d18dc0e06739 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:01.226829 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8913c-a920-4e24-b597-d18dc0e06739 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:01.236064 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 617/1236] 10.222.0.74 () {56 vars in 1103 bytes} [Wed Jul 27 16:26:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:26:01.242544 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b306663-9340-49bb-964d-ea16a4a2e40a None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:01.338958 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:01.357770 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b306663-9340-49bb-964d-ea16a4a2e40a None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:01.396574 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a70078-2981-450e-953e-d89da6dca600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:01.397252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a70078-2981-450e-953e-d89da6dca600 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:01.397532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a70078-2981-450e-953e-d89da6dca600 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:01.397802 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a70078-2981-450e-953e-d89da6dca600 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:01.407906 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a70078-2981-450e-953e-d89da6dca600 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:01.409393 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a70078-2981-450e-953e-d89da6dca600 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:01.424915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-06a70078-2981-450e-953e-d89da6dca600 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:26:01.424915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:01.424915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:01.424915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:01.424915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:01.424915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:01.424915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:01.424915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:01.424915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:01.424915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:01.424915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:01.424915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:01.424915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:26:01.424915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:26:01.424915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:26:01.426421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:26:01.426421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:01.426421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:01.426421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:01.426421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:01.426421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:01.426421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:01.426421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:01.426421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:01.426421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:01.426421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:01.426421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:01.426421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:01.426421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:01.426421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:01.426421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:01.426421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:01.427421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:26:01.427421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:26:01.427421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:26:01.427421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:26:01.427421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 620/1237] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:26:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:26:01.432597 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8913c-a920-4e24-b597-d18dc0e06739 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:01.537168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8913c-a920-4e24-b597-d18dc0e06739 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:01.547178 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:01.573439 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34e424-51f5-4666-9caf-11af427031fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:01.574057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34e424-51f5-4666-9caf-11af427031fe None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:01.574337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34e424-51f5-4666-9caf-11af427031fe None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:01.574607 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34e424-51f5-4666-9caf-11af427031fe None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:01.575289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34e424-51f5-4666-9caf-11af427031fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:01.575590 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34e424-51f5-4666-9caf-11af427031fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:01.576981 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34e424-51f5-4666-9caf-11af427031fe None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:01.593159 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 618/1238] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:01.599247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a70078-2981-450e-953e-d89da6dca600 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:01.707738 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a70078-2981-450e-953e-d89da6dca600 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:01.746144 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232b2d9-7cd5-4edb-8f52-e0fd9532b55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:01.746783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232b2d9-7cd5-4edb-8f52-e0fd9532b55e None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:01.747064 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232b2d9-7cd5-4edb-8f52-e0fd9532b55e None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:01.747349 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232b2d9-7cd5-4edb-8f52-e0fd9532b55e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:01.750111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:01.756279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232b2d9-7cd5-4edb-8f52-e0fd9532b55e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:01.758355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232b2d9-7cd5-4edb-8f52-e0fd9532b55e None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:01.770896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-c232b2d9-7cd5-4edb-8f52-e0fd9532b55e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 16:26:01.770896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:01.770896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:01.770896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:01.770896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:01.770896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:01.770896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:01.770896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:01.770896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:01.770896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:01.770896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:01.770896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:01.770896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:26:01.770896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:26:01.770896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:26:01.770896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:26:01.772582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:01.772582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:01.772582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:26:01.772582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:26:01.772582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:26:01.772582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:26:01.772582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:01.772582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:01.772582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:01.772582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:01.772582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:01.772582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:01.772582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:01.772582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:01.772582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:01.772582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:01.772582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:01.772582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:01.773783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:01.773783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:01.773783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:26:01.773783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:26:01.773783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:26:01.773783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:26:01.773783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:26:01.773783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:26:01.773783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 16:26:01.773783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:26:01.773783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 621/1239] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:26:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:26:01.778216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34e424-51f5-4666-9caf-11af427031fe None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:01.888915 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34e424-51f5-4666-9caf-11af427031fe None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:01.932802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf030e-3248-485d-ade8-17716f8db9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:01.933777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf030e-3248-485d-ade8-17716f8db9fc None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:01.934152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf030e-3248-485d-ade8-17716f8db9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:01.934520 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf030e-3248-485d-ade8-17716f8db9fc None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:01.935385 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf030e-3248-485d-ade8-17716f8db9fc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:01.935753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf030e-3248-485d-ade8-17716f8db9fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:01.937412 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf030e-3248-485d-ade8-17716f8db9fc None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:01.956913 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 619/1240] 10.222.0.74 () {56 vars in 1143 bytes} [Wed Jul 27 16:26:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:26:01.963167 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232b2d9-7cd5-4edb-8f52-e0fd9532b55e None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:02.067941 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232b2d9-7cd5-4edb-8f52-e0fd9532b55e None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:02.106648 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447d537-96f8-4bba-aeb8-2381d88de3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:02.107568 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447d537-96f8-4bba-aeb8-2381d88de3a3 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:02.107568 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447d537-96f8-4bba-aeb8-2381d88de3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:02.107797 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447d537-96f8-4bba-aeb8-2381d88de3a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:02.175292 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:02.185739 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447d537-96f8-4bba-aeb8-2381d88de3a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:02.187856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447d537-96f8-4bba-aeb8-2381d88de3a3 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:02.299545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 622/1241] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:26:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 16:26:02.305993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf030e-3248-485d-ade8-17716f8db9fc None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:02.335377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:02.414613 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf030e-3248-485d-ade8-17716f8db9fc None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:02.453915 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ed008-2bb7-4951-82d6-0a20ae0455df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:02.454767 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ed008-2bb7-4951-82d6-0a20ae0455df None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:02.455054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ed008-2bb7-4951-82d6-0a20ae0455df None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:02.455340 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ed008-2bb7-4951-82d6-0a20ae0455df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:02.468804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ed008-2bb7-4951-82d6-0a20ae0455df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:02.470296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ed008-2bb7-4951-82d6-0a20ae0455df None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:02.485731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-308ed008-2bb7-4951-82d6-0a20ae0455df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:26:02.485731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:02.485731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:02.485731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:02.485731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:02.485731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:02.485731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:02.485731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:02.485731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:02.485731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:02.485731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:02.485731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:02.485731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:26:02.485731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:26:02.485731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:26:02.486886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:26:02.486886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:02.486886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:02.486886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:02.486886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:02.486886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:02.486886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:02.486886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:02.486886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:02.486886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:02.486886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:02.486886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:02.486886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:02.486886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:02.486886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:02.486886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:02.486886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:02.487830 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:26:02.487830 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:26:02.487830 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:26:02.487830 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:26:02.488109 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 620/1242] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:26:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:26:02.494366 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447d537-96f8-4bba-aeb8-2381d88de3a3 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:02.591740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:02.598282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447d537-96f8-4bba-aeb8-2381d88de3a3 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:02.639991 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457e667-d445-4f92-b422-07b0b669d264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:02.640709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457e667-d445-4f92-b422-07b0b669d264 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:02.641027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457e667-d445-4f92-b422-07b0b669d264 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:02.641304 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457e667-d445-4f92-b422-07b0b669d264 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:02.641995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457e667-d445-4f92-b422-07b0b669d264 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:02.642366 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457e667-d445-4f92-b422-07b0b669d264 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:02.643959 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457e667-d445-4f92-b422-07b0b669d264 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:02.656561 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 623/1243] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:02.662236 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ed008-2bb7-4951-82d6-0a20ae0455df None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:02.746516 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:02.769416 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ed008-2bb7-4951-82d6-0a20ae0455df None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:02.806587 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3105f7a-17b5-45d6-b1c9-55e4708d1d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:02.807330 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3105f7a-17b5-45d6-b1c9-55e4708d1d0f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:02.807520 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3105f7a-17b5-45d6-b1c9-55e4708d1d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:02.807814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3105f7a-17b5-45d6-b1c9-55e4708d1d0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:02.815501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3105f7a-17b5-45d6-b1c9-55e4708d1d0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:02.817064 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3105f7a-17b5-45d6-b1c9-55e4708d1d0f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:02.832955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-e3105f7a-17b5-45d6-b1c9-55e4708d1d0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:26:02.832955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:02.832955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:02.832955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:02.832955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:02.832955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:02.832955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:02.832955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:02.832955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:02.832955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:02.832955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:02.832955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:02.832955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:26:02.832955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:26:02.832955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:26:02.834118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:26:02.834118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:02.834118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:02.834118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:02.834118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:02.834118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:02.834118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:02.834118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:02.834118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:02.834118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:02.834118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:02.834118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:02.834118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:02.834118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:02.834118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:02.834118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:02.834118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:02.835154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:26:02.835154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:26:02.835154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:26:02.835154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:26:02.835154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:26:02.835154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:26:02.835154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:26:02.835154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:26:02.835731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 621/1244] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:26:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:26:02.842116 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457e667-d445-4f92-b422-07b0b669d264 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:02.961432 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457e667-d445-4f92-b422-07b0b669d264 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:02.997746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c2a6b-fd0d-4df4-b4ef-efb9911a3571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:02.998329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c2a6b-fd0d-4df4-b4ef-efb9911a3571 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:02.998648 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c2a6b-fd0d-4df4-b4ef-efb9911a3571 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:02.998953 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c2a6b-fd0d-4df4-b4ef-efb9911a3571 None admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:02.999767 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c2a6b-fd0d-4df4-b4ef-efb9911a3571 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:03.000100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c2a6b-fd0d-4df4-b4ef-efb9911a3571 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:03.001688 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c2a6b-fd0d-4df4-b4ef-efb9911a3571 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:03.019441 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 624/1245] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:26:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:26:03.022034 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3105f7a-17b5-45d6-b1c9-55e4708d1d0f None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:03.129985 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3105f7a-17b5-45d6-b1c9-55e4708d1d0f None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:03.165216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c630bb6-06ec-4f88-b41c-57cb782f7f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:03.165843 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c630bb6-06ec-4f88-b41c-57cb782f7f52 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:03.166124 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c630bb6-06ec-4f88-b41c-57cb782f7f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:03.166400 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c630bb6-06ec-4f88-b41c-57cb782f7f52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:03.167207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c630bb6-06ec-4f88-b41c-57cb782f7f52 None admin] RBAC: Adding query filter params (role.id=2aa0cdfed849496db8f3692c9569d7ee, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=91f8e0f82a8541aead6cd8cc45f4a86b) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:03.167831 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c630bb6-06ec-4f88-b41c-57cb782f7f52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:03.168951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c630bb6-06ec-4f88-b41c-57cb782f7f52 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:03.174955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 622/1246] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:26:03 2022] GET /identity/v3/role_assignments?user.id=91f8e0f82a8541aead6cd8cc45f4a86b&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=2aa0cdfed849496db8f3692c9569d7ee => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:05.436650 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d8eb2-b52e-4c9c-90f8-1978fe07681a None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:05.437480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d8eb2-b52e-4c9c-90f8-1978fe07681a None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:05.437480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d8eb2-b52e-4c9c-90f8-1978fe07681a None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:05.438628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 625/1247] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:26:05.444126 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912c7b9-68dc-4a6b-8211-13f24e430821 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:05.445259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912c7b9-68dc-4a6b-8211-13f24e430821 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:05.445377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912c7b9-68dc-4a6b-8211-13f24e430821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:05.452150 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:05.465266 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-9912c7b9-68dc-4a6b-8211-13f24e430821 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:26:05.466538 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9912c7b9-68dc-4a6b-8211-13f24e430821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:26:05.539143 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 623/1248] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:26:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:26:05.563517 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d8eb2-b52e-4c9c-90f8-1978fe07681a None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:05.583626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:05.697717 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d8eb2-b52e-4c9c-90f8-1978fe07681a None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:05.736597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abd688-0c7e-40e5-a506-aa0c268ebf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:05.737289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abd688-0c7e-40e5-a506-aa0c268ebf9f None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:05.737604 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abd688-0c7e-40e5-a506-aa0c268ebf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:05.737967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abd688-0c7e-40e5-a506-aa0c268ebf9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:05.748732 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53abd688-0c7e-40e5-a506-aa0c268ebf9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:26:05.753961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-53abd688-0c7e-40e5-a506-aa0c268ebf9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:26:05.753961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:05.753961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:05.753961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:05.753961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:05.753961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:05.753961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:05.753961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:05.753961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:05.753961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:05.753961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:05.753961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:05.753961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:26:05.753961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:26:05.753961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:26:05.753961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:26:05.755068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:26:05.755068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:26:05.755068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:26:05.755068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:26:05.755068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:05.755068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:05.755068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:05.755068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:05.755068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:05.755068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:05.755068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:05.755068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:05.755068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:05.755068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:05.755068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:05.755068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:05.756028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:05.756028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:05.756028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:05.756028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:05.756028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:26:05.756028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:26:05.756028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:26:05.756028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:26:05.756028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:26:05.756028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:26:05.756028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:26:05.756028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:26:05.756028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 626/1249] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:26:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:26:05.761906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912c7b9-68dc-4a6b-8211-13f24e430821 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:05.877382 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912c7b9-68dc-4a6b-8211-13f24e430821 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:05.917618 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:05.923863 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af037251-90ea-4b42-9dd4-62d5e6503784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:05.924620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af037251-90ea-4b42-9dd4-62d5e6503784 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:05.924877 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af037251-90ea-4b42-9dd4-62d5e6503784 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:05.925214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af037251-90ea-4b42-9dd4-62d5e6503784 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:05.925999 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af037251-90ea-4b42-9dd4-62d5e6503784 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:05.926339 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af037251-90ea-4b42-9dd4-62d5e6503784 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:05.927874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af037251-90ea-4b42-9dd4-62d5e6503784 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:05.942515 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 624/1250] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:26:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:05.943839 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abd688-0c7e-40e5-a506-aa0c268ebf9f None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:06.033368 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:06.059887 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abd688-0c7e-40e5-a506-aa0c268ebf9f None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:06.100371 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2fb3f8-dcd9-4578-a300-3db4ac38c8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:06.101136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fb3f8-dcd9-4578-a300-3db4ac38c8bd None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:06.101447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fb3f8-dcd9-4578-a300-3db4ac38c8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:06.101757 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fb3f8-dcd9-4578-a300-3db4ac38c8bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:06.113173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2fb3f8-dcd9-4578-a300-3db4ac38c8bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:06.114728 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2fb3f8-dcd9-4578-a300-3db4ac38c8bd None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:06.132331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-5c2fb3f8-dcd9-4578-a300-3db4ac38c8bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:26:06.132331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:06.132331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:06.132331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:06.132331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:06.132331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:06.132331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:06.132331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:06.132331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:06.132331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:06.132331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:06.132331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:06.132331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:26:06.132331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:26:06.132331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:26:06.133276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:26:06.133276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:06.133276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:06.133276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:06.133276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:06.133276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:06.133276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:06.133276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:06.133276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:06.133276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:06.133276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:06.133276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:06.133276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:06.133276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:06.133276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:06.133276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:06.133276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:06.134193 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:26:06.134193 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:26:06.134193 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:26:06.134193 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:26:06.134193 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 627/1251] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:26:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:26:06.139785 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af037251-90ea-4b42-9dd4-62d5e6503784 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:06.253457 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af037251-90ea-4b42-9dd4-62d5e6503784 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:06.291920 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d97b4f-ead9-4893-a549-b624173b05c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:06.292451 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d97b4f-ead9-4893-a549-b624173b05c0 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:06.292735 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d97b4f-ead9-4893-a549-b624173b05c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:06.293058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d97b4f-ead9-4893-a549-b624173b05c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:06.293778 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d97b4f-ead9-4893-a549-b624173b05c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:06.294081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d97b4f-ead9-4893-a549-b624173b05c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:06.295454 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d97b4f-ead9-4893-a549-b624173b05c0 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:06.307817 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 625/1252] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:06.313929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2fb3f8-dcd9-4578-a300-3db4ac38c8bd None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:06.430278 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2fb3f8-dcd9-4578-a300-3db4ac38c8bd None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:06.459654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:06.468661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9099b86-6c42-44c2-a060-bc3f883faf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:06.469337 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9099b86-6c42-44c2-a060-bc3f883faf7b None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:06.469619 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9099b86-6c42-44c2-a060-bc3f883faf7b None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:06.469891 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9099b86-6c42-44c2-a060-bc3f883faf7b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:06.477376 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9099b86-6c42-44c2-a060-bc3f883faf7b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:06.478826 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9099b86-6c42-44c2-a060-bc3f883faf7b None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:06.491918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-e9099b86-6c42-44c2-a060-bc3f883faf7b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 16:26:06.491918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:06.491918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:06.491918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:06.491918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:06.491918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:06.491918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:06.491918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:06.491918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:06.491918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:06.491918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:06.491918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:06.491918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:26:06.491918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:26:06.491918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:26:06.491918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:26:06.493120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:06.493120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:06.493120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:26:06.493120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:26:06.493120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:26:06.493120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:26:06.493120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:06.493120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:06.493120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:06.493120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:06.493120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:06.493120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:06.493120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:06.493120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:06.493120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:06.493120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:06.493120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:06.493120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:06.494279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:06.494279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:06.494279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:26:06.494279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:26:06.494279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:26:06.494279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:26:06.494279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:26:06.494279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:26:06.494279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 16:26:06.494279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:26:06.494279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 628/1253] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:26:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:26:06.501030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d97b4f-ead9-4893-a549-b624173b05c0 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:06.565167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:06.619147 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d97b4f-ead9-4893-a549-b624173b05c0 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:06.661862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c332df-6bfb-48f8-85c3-9036a0651f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:06.662666 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c332df-6bfb-48f8-85c3-9036a0651f50 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:06.662939 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c332df-6bfb-48f8-85c3-9036a0651f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:06.663287 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c332df-6bfb-48f8-85c3-9036a0651f50 None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:06.664130 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c332df-6bfb-48f8-85c3-9036a0651f50 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:06.664445 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c332df-6bfb-48f8-85c3-9036a0651f50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:06.666074 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c332df-6bfb-48f8-85c3-9036a0651f50 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:06.685496 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 626/1254] 10.222.0.74 () {56 vars in 1143 bytes} [Wed Jul 27 16:26:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:06.691682 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9099b86-6c42-44c2-a060-bc3f883faf7b None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:06.813897 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9099b86-6c42-44c2-a060-bc3f883faf7b None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:06.853913 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3331d-a847-48f6-97fd-59bdc1b61b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:06.854665 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3331d-a847-48f6-97fd-59bdc1b61b59 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:06.855151 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3331d-a847-48f6-97fd-59bdc1b61b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:06.855471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3331d-a847-48f6-97fd-59bdc1b61b59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:06.939407 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:06.957364 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a3331d-a847-48f6-97fd-59bdc1b61b59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:06.958990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a3331d-a847-48f6-97fd-59bdc1b61b59 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:07.064804 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 629/1255] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:26:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 16:26:07.071157 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c332df-6bfb-48f8-85c3-9036a0651f50 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:07.188688 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c332df-6bfb-48f8-85c3-9036a0651f50 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:07.202724 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:07.226057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f79c29-d251-4933-87f9-a052b41c202a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:07.226705 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f79c29-d251-4933-87f9-a052b41c202a None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:07.226990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f79c29-d251-4933-87f9-a052b41c202a None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:07.227267 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f79c29-d251-4933-87f9-a052b41c202a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:07.237595 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f79c29-d251-4933-87f9-a052b41c202a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:07.239052 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f79c29-d251-4933-87f9-a052b41c202a None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:07.253848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-09f79c29-d251-4933-87f9-a052b41c202a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:26:07.253848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:07.253848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:07.253848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:07.253848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:07.253848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:07.253848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:07.253848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:07.253848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:07.253848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:07.253848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:07.253848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:07.253848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:26:07.253848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:26:07.253848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:26:07.255030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:26:07.255030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:07.255030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:07.255030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:07.255030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:07.255030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:07.255030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:07.255030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:07.255030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:07.255030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:07.255030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:07.255030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:07.255030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:07.255030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:07.255030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:07.255030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:07.255030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:07.255988 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:26:07.255988 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:26:07.255988 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:26:07.255988 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:26:07.256242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 627/1256] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:26:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:26:07.261647 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3331d-a847-48f6-97fd-59bdc1b61b59 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:07.363798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:07.376842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3331d-a847-48f6-97fd-59bdc1b61b59 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:07.414170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db9e50-4cb6-4668-af94-8fb7b8f61820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:07.414791 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db9e50-4cb6-4668-af94-8fb7b8f61820 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:07.415089 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db9e50-4cb6-4668-af94-8fb7b8f61820 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:07.415367 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db9e50-4cb6-4668-af94-8fb7b8f61820 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:07.416101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db9e50-4cb6-4668-af94-8fb7b8f61820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:07.416447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db9e50-4cb6-4668-af94-8fb7b8f61820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:07.417930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db9e50-4cb6-4668-af94-8fb7b8f61820 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:07.431070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 630/1257] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:07.437050 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f79c29-d251-4933-87f9-a052b41c202a None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:07.546056 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f79c29-d251-4933-87f9-a052b41c202a None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:07.581178 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf4513-9c2b-4d7b-a953-a5c4acd2772e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:07.581825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf4513-9c2b-4d7b-a953-a5c4acd2772e None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:07.582110 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf4513-9c2b-4d7b-a953-a5c4acd2772e None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:07.582385 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf4513-9c2b-4d7b-a953-a5c4acd2772e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:07.589401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf4513-9c2b-4d7b-a953-a5c4acd2772e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:07.590865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf4513-9c2b-4d7b-a953-a5c4acd2772e None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:07.602388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-cdaf4513-9c2b-4d7b-a953-a5c4acd2772e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:26:07.602388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:07.602388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:07.602388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:07.602388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:07.602388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:07.602388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:07.602388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:07.602388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:07.602388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:07.602388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:07.602388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:07.602388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:26:07.602388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:26:07.602388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:26:07.603528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:26:07.603528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:07.603528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:07.603528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:07.603528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:07.603528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:07.603528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:07.603528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:07.603528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:07.603528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:07.603528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:07.603528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:07.603528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:07.603528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:07.603528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:07.603528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:07.603528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:07.604437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:26:07.604437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:26:07.604437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:26:07.604437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:26:07.604437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:26:07.604437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:26:07.604437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:26:07.604437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:26:07.604437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 628/1258] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:26:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 16:26:07.609857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db9e50-4cb6-4668-af94-8fb7b8f61820 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:07.719313 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db9e50-4cb6-4668-af94-8fb7b8f61820 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:07.755548 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c792fa-b087-444f-910c-35c774359b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:07.756245 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c792fa-b087-444f-910c-35c774359b51 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:07.756540 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c792fa-b087-444f-910c-35c774359b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:07.756807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c792fa-b087-444f-910c-35c774359b51 None admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:07.757607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c792fa-b087-444f-910c-35c774359b51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:07.757917 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c792fa-b087-444f-910c-35c774359b51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:07.759349 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c792fa-b087-444f-910c-35c774359b51 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:07.771562 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 631/1259] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:26:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:07.778146 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf4513-9c2b-4d7b-a953-a5c4acd2772e None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:07.807511 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:07.888089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf4513-9c2b-4d7b-a953-a5c4acd2772e None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:07.936663 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b0ed7-8328-4cfd-90a6-1f84e70260fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:07.937374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b0ed7-8328-4cfd-90a6-1f84e70260fc None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:07.937753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b0ed7-8328-4cfd-90a6-1f84e70260fc None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:07.938129 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b0ed7-8328-4cfd-90a6-1f84e70260fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:07.939026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b0ed7-8328-4cfd-90a6-1f84e70260fc None admin] RBAC: Adding query filter params (role.id=ca37e43082fa40c6a88b12680a0ca94b, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=91f8e0f82a8541aead6cd8cc45f4a86b) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:07.939478 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b0ed7-8328-4cfd-90a6-1f84e70260fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:07.941237 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b0ed7-8328-4cfd-90a6-1f84e70260fc None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:07.949771 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 629/1260] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:26:07 2022] GET /identity/v3/role_assignments?user.id=91f8e0f82a8541aead6cd8cc45f4a86b&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=ca37e43082fa40c6a88b12680a0ca94b => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:09.939860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124024a8-a5b4-4e60-97f0-c4468858b1b9 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:09.940414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124024a8-a5b4-4e60-97f0-c4468858b1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:09.940414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124024a8-a5b4-4e60-97f0-c4468858b1b9 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:09.941799 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 632/1261] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:26:09.946819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967dfa33-6f15-48c8-b78a-7c487973aa44 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:09.947242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967dfa33-6f15-48c8-b78a-7c487973aa44 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:09.947378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967dfa33-6f15-48c8-b78a-7c487973aa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:09.965801 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-967dfa33-6f15-48c8-b78a-7c487973aa44 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:26:09.966995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-967dfa33-6f15-48c8-b78a-7c487973aa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:26:10.031145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 630/1262] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:26:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:26:10.052622 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124024a8-a5b4-4e60-97f0-c4468858b1b9 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:10.063811 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:10.152325 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124024a8-a5b4-4e60-97f0-c4468858b1b9 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:10.188452 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7bbd2-5e03-4938-bf62-50af51912e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:10.189118 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7bbd2-5e03-4938-bf62-50af51912e73 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:10.189400 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7bbd2-5e03-4938-bf62-50af51912e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:10.189677 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7bbd2-5e03-4938-bf62-50af51912e73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:10.199509 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f7bbd2-5e03-4938-bf62-50af51912e73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:26:10.204697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-83f7bbd2-5e03-4938-bf62-50af51912e73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:26:10.204697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:10.204697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:10.204697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:10.204697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:10.204697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:10.204697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:10.204697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:10.204697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:10.204697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:10.204697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:10.204697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:10.204697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:26:10.204697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:26:10.204697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:26:10.204697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:26:10.205614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:26:10.205614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:26:10.205614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:26:10.205614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:26:10.205614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:10.205614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:10.205614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:10.205614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:10.205614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:10.205614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:10.205614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:10.205614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:10.205614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:10.205614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:10.205614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:10.205614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:10.206451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:10.206451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:10.206451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:10.206451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:10.206451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:26:10.206451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:26:10.206451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:26:10.206451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:26:10.206451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:26:10.206451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:26:10.206451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:26:10.206451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:26:10.206451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 633/1263] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:26:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:26:10.211874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967dfa33-6f15-48c8-b78a-7c487973aa44 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:10.229981 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:10.308642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967dfa33-6f15-48c8-b78a-7c487973aa44 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:10.341617 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36da9d-2ca0-4698-bc88-247c78efb0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:10.342218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36da9d-2ca0-4698-bc88-247c78efb0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:10.342494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36da9d-2ca0-4698-bc88-247c78efb0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:10.342765 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36da9d-2ca0-4698-bc88-247c78efb0e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:10.343468 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36da9d-2ca0-4698-bc88-247c78efb0e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:10.343766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36da9d-2ca0-4698-bc88-247c78efb0e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:10.345159 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36da9d-2ca0-4698-bc88-247c78efb0e9 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:10.353390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 631/1264] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:26:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:10.359064 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7bbd2-5e03-4938-bf62-50af51912e73 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:10.424243 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:10.457061 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7bbd2-5e03-4938-bf62-50af51912e73 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:10.489576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31317f-1275-4695-b03e-03bab307d7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:10.490196 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31317f-1275-4695-b03e-03bab307d7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:10.490475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31317f-1275-4695-b03e-03bab307d7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:10.490748 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31317f-1275-4695-b03e-03bab307d7f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:10.500995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31317f-1275-4695-b03e-03bab307d7f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:10.502426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31317f-1275-4695-b03e-03bab307d7f1 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:10.516626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-0d31317f-1275-4695-b03e-03bab307d7f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:26:10.516626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:10.516626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:10.516626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:10.516626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:10.516626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:10.516626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:10.516626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:10.516626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:10.516626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:10.516626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:10.516626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:10.516626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:26:10.516626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:26:10.516626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:26:10.517672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:26:10.517672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:10.517672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:10.517672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:10.517672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:10.517672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:10.517672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:10.517672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:10.517672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:10.517672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:10.517672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:10.517672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:10.517672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:10.517672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:10.517672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:10.517672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:10.517672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:10.518596 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:26:10.518596 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:26:10.518596 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:26:10.518596 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:26:10.518596 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 634/1265] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:26:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:26:10.524042 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36da9d-2ca0-4698-bc88-247c78efb0e9 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:10.602039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:10.620675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36da9d-2ca0-4698-bc88-247c78efb0e9 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:10.652629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b4e4e-708a-4008-b640-2528f7207f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:10.653255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b4e4e-708a-4008-b640-2528f7207f93 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:10.653537 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b4e4e-708a-4008-b640-2528f7207f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:10.653812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b4e4e-708a-4008-b640-2528f7207f93 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:10.654488 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b4e4e-708a-4008-b640-2528f7207f93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:10.654789 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b4e4e-708a-4008-b640-2528f7207f93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:10.656151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b4e4e-708a-4008-b640-2528f7207f93 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:10.668106 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 632/1266] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:10.673712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31317f-1275-4695-b03e-03bab307d7f1 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:10.771425 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31317f-1275-4695-b03e-03bab307d7f1 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:10.791378 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:10.805456 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7c380-7652-4d03-a77e-652a709ac213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:10.806085 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7c380-7652-4d03-a77e-652a709ac213 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:10.806368 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7c380-7652-4d03-a77e-652a709ac213 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:10.806644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7c380-7652-4d03-a77e-652a709ac213 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:10.814630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a7c380-7652-4d03-a77e-652a709ac213 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:10.816073 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a7c380-7652-4d03-a77e-652a709ac213 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:10.827100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-88a7c380-7652-4d03-a77e-652a709ac213 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 16:26:10.827100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:10.827100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:10.827100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:10.827100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:10.827100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:10.827100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:10.827100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:10.827100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:10.827100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:10.827100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:10.827100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:10.827100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:26:10.827100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:26:10.827100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:26:10.827100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:26:10.828447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:10.828447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:10.828447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:26:10.828447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:26:10.828447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:26:10.828447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:26:10.828447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:10.828447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:10.828447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:10.828447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:10.828447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:10.828447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:10.828447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:10.828447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:10.828447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:10.828447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:10.828447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:10.828447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:10.829718 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:10.829718 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:10.829718 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:26:10.829718 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:26:10.829718 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:26:10.829718 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:26:10.829718 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:26:10.829718 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:26:10.829718 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 16:26:10.829718 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:26:10.829718 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 635/1267] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:26:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:26:10.834364 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b4e4e-708a-4008-b640-2528f7207f93 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:10.934293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b4e4e-708a-4008-b640-2528f7207f93 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:10.970863 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c199305-1c04-4189-b16b-f42f9fa345f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:10.971477 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c199305-1c04-4189-b16b-f42f9fa345f9 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:10.971747 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c199305-1c04-4189-b16b-f42f9fa345f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:10.972021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c199305-1c04-4189-b16b-f42f9fa345f9 None admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:10.972806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c199305-1c04-4189-b16b-f42f9fa345f9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:10.973143 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c199305-1c04-4189-b16b-f42f9fa345f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:10.974578 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c199305-1c04-4189-b16b-f42f9fa345f9 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:10.983128 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:10.991807 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 633/1268] 10.222.0.74 () {56 vars in 1143 bytes} [Wed Jul 27 16:26:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:10.997573 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7c380-7652-4d03-a77e-652a709ac213 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:11.095283 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7c380-7652-4d03-a77e-652a709ac213 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:11.127217 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b073b-e5a2-4ee9-8d40-e41d7fcc1745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:11.127836 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b073b-e5a2-4ee9-8d40-e41d7fcc1745 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:11.128118 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b073b-e5a2-4ee9-8d40-e41d7fcc1745 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:11.128396 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b073b-e5a2-4ee9-8d40-e41d7fcc1745 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:11.139169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b073b-e5a2-4ee9-8d40-e41d7fcc1745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:11.140604 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b073b-e5a2-4ee9-8d40-e41d7fcc1745 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:11.155103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-946b073b-e5a2-4ee9-8d40-e41d7fcc1745 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:26:11.155103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:11.155103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:11.155103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:11.155103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:11.155103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:11.155103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:11.155103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:11.155103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:11.155103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:11.155103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:11.155103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:11.155103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:26:11.155103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:26:11.155103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:26:11.156054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:26:11.156054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:11.156054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:11.156054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:11.156054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:11.156054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:11.156054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:11.156054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:11.156054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:11.156054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:11.156054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:11.156054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:11.156054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:11.156054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:11.156054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:11.156054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:11.156054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:11.157004 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:26:11.157004 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:26:11.157004 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:26:11.157004 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:26:11.157004 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 636/1269] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:26:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:26:11.162024 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c199305-1c04-4189-b16b-f42f9fa345f9 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:11.259792 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c199305-1c04-4189-b16b-f42f9fa345f9 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:11.292589 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770fbfb-5a98-4568-84ec-4cc1bf4b58ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:11.293226 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770fbfb-5a98-4568-84ec-4cc1bf4b58ea None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:11.293510 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770fbfb-5a98-4568-84ec-4cc1bf4b58ea None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:11.293787 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770fbfb-5a98-4568-84ec-4cc1bf4b58ea None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:11.294501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770fbfb-5a98-4568-84ec-4cc1bf4b58ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:11.294808 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770fbfb-5a98-4568-84ec-4cc1bf4b58ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:11.296176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770fbfb-5a98-4568-84ec-4cc1bf4b58ea None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:11.307616 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 634/1270] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:11.313175 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b073b-e5a2-4ee9-8d40-e41d7fcc1745 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:11.344219 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:11.412928 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b073b-e5a2-4ee9-8d40-e41d7fcc1745 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:11.446377 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652e6a0-cda0-426c-8eb0-aa3b805daa4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:11.447002 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652e6a0-cda0-426c-8eb0-aa3b805daa4c None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:11.447284 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652e6a0-cda0-426c-8eb0-aa3b805daa4c None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:11.447560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652e6a0-cda0-426c-8eb0-aa3b805daa4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:11.454349 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652e6a0-cda0-426c-8eb0-aa3b805daa4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:11.455775 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652e6a0-cda0-426c-8eb0-aa3b805daa4c None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:11.467888 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-0652e6a0-cda0-426c-8eb0-aa3b805daa4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:26:11.467888 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:11.467888 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:11.467888 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:11.467888 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:11.467888 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:11.467888 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:11.467888 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:11.467888 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:11.467888 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:11.467888 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:11.467888 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:11.467888 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:26:11.467888 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:26:11.467888 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:26:11.468934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:26:11.468934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:11.468934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:11.468934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:11.468934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:11.468934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:11.468934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:11.468934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:11.468934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:11.468934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:11.468934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:11.468934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:11.468934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:11.468934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:11.468934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:11.468934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:11.468934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:11.470050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:26:11.470050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:26:11.470050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:26:11.470050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:26:11.470050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:26:11.470050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:26:11.470050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:26:11.470050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:26:11.470050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 637/1271] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:26:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:26:11.474880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770fbfb-5a98-4568-84ec-4cc1bf4b58ea None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:11.531696 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:11.572791 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770fbfb-5a98-4568-84ec-4cc1bf4b58ea None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:11.606197 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277fd74-b07b-4816-8d01-2dbd17edf15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:11.606801 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277fd74-b07b-4816-8d01-2dbd17edf15f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:11.607084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277fd74-b07b-4816-8d01-2dbd17edf15f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:11.607361 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277fd74-b07b-4816-8d01-2dbd17edf15f None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:11.608097 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277fd74-b07b-4816-8d01-2dbd17edf15f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:11.608404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277fd74-b07b-4816-8d01-2dbd17edf15f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:11.609856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277fd74-b07b-4816-8d01-2dbd17edf15f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:11.621336 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 635/1272] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:26:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:11.627119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652e6a0-cda0-426c-8eb0-aa3b805daa4c None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:11.715488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:11.724184 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652e6a0-cda0-426c-8eb0-aa3b805daa4c None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:11.755730 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1d4b4-220e-4a9b-91f4-9306785f75ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:11.756379 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1d4b4-220e-4a9b-91f4-9306785f75ea None admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:11.757205 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1d4b4-220e-4a9b-91f4-9306785f75ea None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:11.757478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1d4b4-220e-4a9b-91f4-9306785f75ea None admin] PATH_INFO: `/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/91f8e0f82a8541aead6cd8cc45f4a86b/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:11.759367 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1d4b4-220e-4a9b-91f4-9306785f75ea None admin] RBAC: Authorizing `identity:create_grant(project_id=8d6bcdd09cf24a5c8003951db37020cf, user_id=91f8e0f82a8541aead6cd8cc45f4a86b, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:11.760780 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1d4b4-220e-4a9b-91f4-9306785f75ea None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:11.773709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 638/1273] 10.222.0.74 () {58 vars in 1439 bytes} [Wed Jul 27 16:26:11 2022] PUT /identity/v3/projects/8d6bcdd09cf24a5c8003951db37020cf/users/91f8e0f82a8541aead6cd8cc45f4a86b/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:26:13.757052 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cb58d-9f54-4fd4-a605-674213a2c5ad None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:13.757575 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cb58d-9f54-4fd4-a605-674213a2c5ad None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:13.757575 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cb58d-9f54-4fd4-a605-674213a2c5ad None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:13.758933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 636/1274] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:26:13.763976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f00e97-3615-464f-b1e2-0c2476e547a5 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:13.764381 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f00e97-3615-464f-b1e2-0c2476e547a5 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:13.764542 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f00e97-3615-464f-b1e2-0c2476e547a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:13.782963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-66f00e97-3615-464f-b1e2-0c2476e547a5 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:26:13.784139 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-66f00e97-3615-464f-b1e2-0c2476e547a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:26:13.849833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 639/1275] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:26:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:26:13.873361 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6cb58d-9f54-4fd4-a605-674213a2c5ad None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:13.989295 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6cb58d-9f54-4fd4-a605-674213a2c5ad None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:14.005867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:14.023612 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a1182-bca8-4779-adda-45cd1202603b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:14.024193 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a1182-bca8-4779-adda-45cd1202603b None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:14.024475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a1182-bca8-4779-adda-45cd1202603b None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:14.024749 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a1182-bca8-4779-adda-45cd1202603b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:14.034592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a1182-bca8-4779-adda-45cd1202603b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 16:26:14.039532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-142a1182-bca8-4779-adda-45cd1202603b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:26:14.039532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:14.039532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:14.039532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:14.039532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:14.039532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:14.039532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:14.039532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:14.039532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:14.039532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:14.039532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:14.039532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:14.039532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 16:26:14.039532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 16:26:14.039532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 16:26:14.039532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 16:26:14.040611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 16:26:14.040611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 16:26:14.040611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 16:26:14.040611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 16:26:14.040611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:14.040611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:14.040611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:14.040611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:14.040611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:14.040611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:14.040611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:14.040611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:14.040611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:14.040611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:14.040611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:14.040611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:14.041697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:14.041697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:14.041697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:14.041697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:14.041697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 16:26:14.041697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 16:26:14.041697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 16:26:14.041697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 16:26:14.041697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 16:26:14.041697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 16:26:14.041697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 16:26:14.041697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:26:14.041697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 637/1276] 10.222.0.74 () {56 vars in 1096 bytes} [Wed Jul 27 16:26:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:26:14.046739 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f00e97-3615-464f-b1e2-0c2476e547a5 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:14.125841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:14.144470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f00e97-3615-464f-b1e2-0c2476e547a5 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:14.178036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f769cb7-a1d3-4a8d-a97d-de08a2e1246f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:14.178648 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f769cb7-a1d3-4a8d-a97d-de08a2e1246f None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:14.178923 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f769cb7-a1d3-4a8d-a97d-de08a2e1246f None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:14.179190 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f769cb7-a1d3-4a8d-a97d-de08a2e1246f None admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:14.179888 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f769cb7-a1d3-4a8d-a97d-de08a2e1246f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:14.180185 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f769cb7-a1d3-4a8d-a97d-de08a2e1246f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:14.181573 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f769cb7-a1d3-4a8d-a97d-de08a2e1246f None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:14.189295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 640/1277] 10.222.0.74 () {56 vars in 1099 bytes} [Wed Jul 27 16:26:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:14.195435 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a1182-bca8-4779-adda-45cd1202603b None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:14.290787 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a1182-bca8-4779-adda-45cd1202603b None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:14.323805 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195276ef-e838-4a97-bcd8-9879ed543968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:14.324431 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195276ef-e838-4a97-bcd8-9879ed543968 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:14.324708 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195276ef-e838-4a97-bcd8-9879ed543968 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:14.325005 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195276ef-e838-4a97-bcd8-9879ed543968 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:14.335361 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195276ef-e838-4a97-bcd8-9879ed543968 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:14.336804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195276ef-e838-4a97-bcd8-9879ed543968 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:14.350448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-195276ef-e838-4a97-bcd8-9879ed543968 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:26:14.350448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:14.350448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:14.350448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:14.350448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:14.350448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:14.350448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:14.350448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:14.350448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:14.350448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:14.350448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:14.350448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:14.350448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:26:14.350448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:26:14.350448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:26:14.351501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:26:14.351501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:14.351501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:14.351501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:14.351501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:14.351501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:14.351501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:14.351501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:14.351501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:14.351501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:14.351501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:14.351501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:14.351501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:14.351501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:14.351501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:14.351501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:14.351501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:14.352602 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:26:14.352602 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:26:14.352602 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:26:14.352602 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:26:14.352602 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 638/1278] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:26:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:26:14.357276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f769cb7-a1d3-4a8d-a97d-de08a2e1246f None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:14.456419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f769cb7-a1d3-4a8d-a97d-de08a2e1246f None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:14.488283 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302dd30-cbe4-401b-b1e2-36f47cf08d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:14.488904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302dd30-cbe4-401b-b1e2-36f47cf08d01 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:14.489195 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302dd30-cbe4-401b-b1e2-36f47cf08d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:14.489471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302dd30-cbe4-401b-b1e2-36f47cf08d01 None admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:14.490155 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302dd30-cbe4-401b-b1e2-36f47cf08d01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:14.490460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302dd30-cbe4-401b-b1e2-36f47cf08d01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:14.492315 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302dd30-cbe4-401b-b1e2-36f47cf08d01 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:14.503620 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 641/1279] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:14.509777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195276ef-e838-4a97-bcd8-9879ed543968 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:14.539071 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:14.605972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195276ef-e838-4a97-bcd8-9879ed543968 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:14.638948 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cbcc7-a17a-492c-b1d6-5f242d9a1fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:14.639574 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cbcc7-a17a-492c-b1d6-5f242d9a1fef None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:14.639856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cbcc7-a17a-492c-b1d6-5f242d9a1fef None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:14.640131 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cbcc7-a17a-492c-b1d6-5f242d9a1fef None admin] PATH_INFO: `/v3/users/placement` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:14.647333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3cbcc7-a17a-492c-b1d6-5f242d9a1fef None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:14.648778 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3cbcc7-a17a-492c-b1d6-5f242d9a1fef None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:14.659802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-9d3cbcc7-a17a-492c-b1d6-5f242d9a1fef None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 16:26:14.659802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:14.659802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:14.659802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:14.659802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:14.659802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:14.659802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:14.659802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:14.659802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:14.659802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:14.659802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:14.659802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:14.659802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 16:26:14.659802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 16:26:14.659802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 16:26:14.659802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 16:26:14.660906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:14.660906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:14.660906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 16:26:14.660906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:26:14.660906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 16:26:14.660906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 16:26:14.660906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:14.660906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:14.660906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:14.660906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:14.660906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:14.660906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:14.660906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:14.660906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:14.660906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:14.660906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:14.660906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:14.660906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:14.662119 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:14.662119 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:14.662119 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 16:26:14.662119 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 16:26:14.662119 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 16:26:14.662119 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 16:26:14.662119 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 16:26:14.662119 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 16:26:14.662119 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 16:26:14.662119 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:26:14.662119 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 639/1280] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:26:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:26:14.667162 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302dd30-cbe4-401b-b1e2-36f47cf08d01 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:14.670860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:14.762964 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302dd30-cbe4-401b-b1e2-36f47cf08d01 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:14.795396 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13b0f9-56ee-4643-b01e-abbcd2a290a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:14.796003 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13b0f9-56ee-4643-b01e-abbcd2a290a0 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:14.796283 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13b0f9-56ee-4643-b01e-abbcd2a290a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:14.796554 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13b0f9-56ee-4643-b01e-abbcd2a290a0 None admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:14.797361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b13b0f9-56ee-4643-b01e-abbcd2a290a0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:14.797663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b13b0f9-56ee-4643-b01e-abbcd2a290a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:14.799070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b13b0f9-56ee-4643-b01e-abbcd2a290a0 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:14.815485 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 642/1281] 10.222.0.74 () {56 vars in 1143 bytes} [Wed Jul 27 16:26:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:14.821677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cbcc7-a17a-492c-b1d6-5f242d9a1fef None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:14.909771 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:14.920494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cbcc7-a17a-492c-b1d6-5f242d9a1fef None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:14.953566 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1764ae5-eeed-4662-b5e7-4737117dc364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:14.954177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1764ae5-eeed-4662-b5e7-4737117dc364 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:14.954464 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1764ae5-eeed-4662-b5e7-4737117dc364 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:14.954742 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1764ae5-eeed-4662-b5e7-4737117dc364 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:15.020180 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1764ae5-eeed-4662-b5e7-4737117dc364 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:15.021652 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1764ae5-eeed-4662-b5e7-4737117dc364 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:15.105892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:15.116496 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 640/1282] 10.222.0.74 () {58 vars in 1282 bytes} [Wed Jul 27 16:26:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 16:26:15.122284 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13b0f9-56ee-4643-b01e-abbcd2a290a0 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:15.181759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:15.219480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13b0f9-56ee-4643-b01e-abbcd2a290a0 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:15.251865 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1cf25-ca47-4784-bf17-a070af37faa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:15.252485 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1cf25-ca47-4784-bf17-a070af37faa4 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:15.252760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1cf25-ca47-4784-bf17-a070af37faa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:15.253060 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1cf25-ca47-4784-bf17-a070af37faa4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:15.262755 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1cf25-ca47-4784-bf17-a070af37faa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:15.264169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1cf25-ca47-4784-bf17-a070af37faa4 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:15.278795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-2ee1cf25-ca47-4784-bf17-a070af37faa4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:26:15.278795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:15.278795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:15.278795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:15.278795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:15.278795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:15.278795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:15.278795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:15.278795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:15.278795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:15.278795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:15.278795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:15.278795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 16:26:15.278795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 16:26:15.278795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 16:26:15.279935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 16:26:15.279935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:15.279935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:15.279935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:15.279935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:15.279935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:15.279935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:15.279935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:15.279935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:15.279935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:15.279935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:15.279935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:15.279935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:15.279935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:15.279935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:15.279935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:15.279935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:15.280834 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 16:26:15.280834 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 16:26:15.280834 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 16:26:15.280834 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:26:15.280834 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 643/1283] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:26:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:26:15.285600 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1764ae5-eeed-4662-b5e7-4737117dc364 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:15.382084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1764ae5-eeed-4662-b5e7-4737117dc364 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:15.415008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd91c3-8970-40a2-b9d0-1fe505a36d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:15.415617 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd91c3-8970-40a2-b9d0-1fe505a36d20 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:15.415899 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd91c3-8970-40a2-b9d0-1fe505a36d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:15.416172 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd91c3-8970-40a2-b9d0-1fe505a36d20 None admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:15.416964 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd91c3-8970-40a2-b9d0-1fe505a36d20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:15.417224 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd91c3-8970-40a2-b9d0-1fe505a36d20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:15.418593 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bd91c3-8970-40a2-b9d0-1fe505a36d20 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:15.429531 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 641/1284] 10.222.0.74 () {56 vars in 1109 bytes} [Wed Jul 27 16:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:15.434816 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1cf25-ca47-4784-bf17-a070af37faa4 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:15.532471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1cf25-ca47-4784-bf17-a070af37faa4 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:15.545448 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:15.565219 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c685f74-2dfd-401f-b6c9-f290b602ed42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:15.565847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c685f74-2dfd-401f-b6c9-f290b602ed42 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:15.566125 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c685f74-2dfd-401f-b6c9-f290b602ed42 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:15.566397 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c685f74-2dfd-401f-b6c9-f290b602ed42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:15.573320 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c685f74-2dfd-401f-b6c9-f290b602ed42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:15.574760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c685f74-2dfd-401f-b6c9-f290b602ed42 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:15.587445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application [None req-2c685f74-2dfd-401f-b6c9-f290b602ed42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:26:15.587445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:15.587445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:15.587445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:15.587445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:15.587445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:15.587445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:15.587445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:15.587445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:15.587445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:15.587445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:15.587445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:15.587445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 16:26:15.587445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 16:26:15.587445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 16:26:15.588469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 16:26:15.588469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:15.588469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:15.588469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:15.588469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:15.588469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:15.588469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:15.588469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:15.588469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:15.588469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:15.588469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:15.588469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:15.588469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:15.588469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:15.588469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:15.588469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:15.588469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:15.589465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 16:26:15.589465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 16:26:15.589465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 16:26:15.589465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 16:26:15.589465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 16:26:15.589465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 16:26:15.589465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 16:26:15.589465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: ERROR keystone.server.flask.application Jul 27 16:26:15.589465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 644/1285] 10.222.0.74 () {56 vars in 1111 bytes} [Wed Jul 27 16:26:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:26:15.594352 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd91c3-8970-40a2-b9d0-1fe505a36d20 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:15.642184 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:15.690495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd91c3-8970-40a2-b9d0-1fe505a36d20 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:15.723143 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3240ed99-0adf-4f46-95a5-77e1a756ee48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:15.723743 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240ed99-0adf-4f46-95a5-77e1a756ee48 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:15.724026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240ed99-0adf-4f46-95a5-77e1a756ee48 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:15.724297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240ed99-0adf-4f46-95a5-77e1a756ee48 None admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:15.725060 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3240ed99-0adf-4f46-95a5-77e1a756ee48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:15.725388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3240ed99-0adf-4f46-95a5-77e1a756ee48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:15.726804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3240ed99-0adf-4f46-95a5-77e1a756ee48 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:15.738297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 642/1286] 10.222.0.74 () {56 vars in 1148 bytes} [Wed Jul 27 16:26:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:26:15.744850 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c685f74-2dfd-401f-b6c9-f290b602ed42 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:15.840569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c685f74-2dfd-401f-b6c9-f290b602ed42 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:15.873443 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b6a17-34b1-43f4-8d4a-34d0cfa0f397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:15.874028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b6a17-34b1-43f4-8d4a-34d0cfa0f397 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:15.874302 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b6a17-34b1-43f4-8d4a-34d0cfa0f397 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:15.874572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b6a17-34b1-43f4-8d4a-34d0cfa0f397 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:15.875333 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b6a17-34b1-43f4-8d4a-34d0cfa0f397 None admin] RBAC: Adding query filter params (role.id=ca37e43082fa40c6a88b12680a0ca94b, scope.project.id=8d6bcdd09cf24a5c8003951db37020cf, user.id=91f8e0f82a8541aead6cd8cc45f4a86b) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:15.875635 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b6a17-34b1-43f4-8d4a-34d0cfa0f397 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:15.877058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b6a17-34b1-43f4-8d4a-34d0cfa0f397 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:15.883272 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 645/1287] 10.222.0.74 () {56 vars in 1374 bytes} [Wed Jul 27 16:26:15 2022] GET /identity/v3/role_assignments?user.id=91f8e0f82a8541aead6cd8cc45f4a86b&scope.project.id=8d6bcdd09cf24a5c8003951db37020cf&role.id=ca37e43082fa40c6a88b12680a0ca94b => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:17.882617 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba94f4-77f5-478b-9c7f-9b2695334986 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:17.883167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba94f4-77f5-478b-9c7f-9b2695334986 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:17.883167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba94f4-77f5-478b-9c7f-9b2695334986 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:17.884495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 643/1288] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:26:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:26:17.889482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97767b0c-f599-4eec-a4ea-71ed340a0df8 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:17.889854 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97767b0c-f599-4eec-a4ea-71ed340a0df8 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:17.890028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97767b0c-f599-4eec-a4ea-71ed340a0df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:17.909948 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-97767b0c-f599-4eec-a4ea-71ed340a0df8 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:26:17.911181 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-97767b0c-f599-4eec-a4ea-71ed340a0df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:26:17.944378 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:17.983763 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 646/1289] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:26:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 16:26:18.005582 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba94f4-77f5-478b-9c7f-9b2695334986 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:18.102820 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba94f4-77f5-478b-9c7f-9b2695334986 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:18.113139 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:18.137058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8d249-cad0-491f-9fb1-f6284b21fe4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:18.137628 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8d249-cad0-491f-9fb1-f6284b21fe4b None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:18.137912 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8d249-cad0-491f-9fb1-f6284b21fe4b None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:18.138187 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8d249-cad0-491f-9fb1-f6284b21fe4b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:18.138808 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b8d249-cad0-491f-9fb1-f6284b21fe4b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:18.140177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b8d249-cad0-491f-9fb1-f6284b21fe4b None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:18.148950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-66b8d249-cad0-491f-9fb1-f6284b21fe4b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 16:26:18.148950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:18.148950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:26:18.148950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:26:18.148950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:26:18.148950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:26:18.148950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:26:18.148950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:26:18.148950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 16:26:18.148950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:26:18.148950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:26:18.148950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:26:18.148950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:18.148950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:18.148950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:18.148950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:18.148950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:18.150007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:18.150007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:18.150007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:18.150007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:18.150007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:18.150007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:18.150007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:26:18.150007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:26:18.150007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:26:18.150007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:26:18.150007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:18.150007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:18.150007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:18.150007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:18.150007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:18.150007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:18.150865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:18.150865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:18.150865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:18.150865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:18.150865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:18.150865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:18.150865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:18.150865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:18.150865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:18.150865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:18.150865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:26:18.150865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:26:18.150865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 16:26:18.150865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:26:18.150865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 644/1290] 10.222.0.74 () {56 vars in 1117 bytes} [Wed Jul 27 16:26:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:26:18.156116 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97767b0c-f599-4eec-a4ea-71ed340a0df8 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:18.252717 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97767b0c-f599-4eec-a4ea-71ed340a0df8 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:18.285660 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8a9c0-5d05-4d03-919d-1af6bd7e7f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:18.286266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8a9c0-5d05-4d03-919d-1af6bd7e7f40 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:18.286551 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8a9c0-5d05-4d03-919d-1af6bd7e7f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:18.286827 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8a9c0-5d05-4d03-919d-1af6bd7e7f40 None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:18.287498 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8a9c0-5d05-4d03-919d-1af6bd7e7f40 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:18.287828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8a9c0-5d05-4d03-919d-1af6bd7e7f40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:18.289219 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8a9c0-5d05-4d03-919d-1af6bd7e7f40 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:18.293905 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 647/1291] 10.222.0.74 () {56 vars in 1116 bytes} [Wed Jul 27 16:26:18 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:18.299455 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8d249-cad0-491f-9fb1-f6284b21fe4b None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:18.395855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8d249-cad0-491f-9fb1-f6284b21fe4b None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:18.427605 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ca259-935e-4bcb-9ca2-f0d4e3d2a44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:18.428773 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ca259-935e-4bcb-9ca2-f0d4e3d2a44f None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:18.429090 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ca259-935e-4bcb-9ca2-f0d4e3d2a44f None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:18.429364 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ca259-935e-4bcb-9ca2-f0d4e3d2a44f None admin] PATH_INFO: `/v3/services` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:18.430024 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ca259-935e-4bcb-9ca2-f0d4e3d2a44f None admin] RBAC: Adding query filter params (type=placement) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:18.430324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ca259-935e-4bcb-9ca2-f0d4e3d2a44f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:18.431672 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ca259-935e-4bcb-9ca2-f0d4e3d2a44f None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:18.435944 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 645/1292] 10.222.0.74 () {56 vars in 1116 bytes} [Wed Jul 27 16:26:18 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:20.378462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0b9d2-9985-471d-97a6-3a7979d3cb6d None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:20.378977 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0b9d2-9985-471d-97a6-3a7979d3cb6d None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:20.379064 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0b9d2-9985-471d-97a6-3a7979d3cb6d None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:20.380346 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 648/1293] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:26:20.385485 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8609fe0-d4c3-49e4-a117-5885f43ea87b None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:20.385868 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8609fe0-d4c3-49e4-a117-5885f43ea87b None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:20.386041 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8609fe0-d4c3-49e4-a117-5885f43ea87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:20.404289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-d8609fe0-d4c3-49e4-a117-5885f43ea87b None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:26:20.405502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d8609fe0-d4c3-49e4-a117-5885f43ea87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:26:20.435236 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:20.471812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 646/1294] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:26:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:26:20.494925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0b9d2-9985-471d-97a6-3a7979d3cb6d None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:20.525349 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:20.593719 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0b9d2-9985-471d-97a6-3a7979d3cb6d None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:20.626292 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565add8d-cd2a-4e32-91ee-45dda4f601e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:20.626894 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565add8d-cd2a-4e32-91ee-45dda4f601e0 None admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:20.627173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565add8d-cd2a-4e32-91ee-45dda4f601e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:20.627448 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565add8d-cd2a-4e32-91ee-45dda4f601e0 None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:20.628136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565add8d-cd2a-4e32-91ee-45dda4f601e0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:20.629515 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565add8d-cd2a-4e32-91ee-45dda4f601e0 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:20.643775 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 649/1295] 10.222.0.74 () {60 vars in 1141 bytes} [Wed Jul 27 16:26:20 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:26:22.622606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a9869-f5dc-4236-adb9-9d23609c608a None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:22.623195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a9869-f5dc-4236-adb9-9d23609c608a None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:22.623195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a9869-f5dc-4236-adb9-9d23609c608a None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:22.624505 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 647/1296] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:26:22.629666 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9f704-3ecf-497a-8c90-39ff4c49f445 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:22.630055 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9f704-3ecf-497a-8c90-39ff4c49f445 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:22.630231 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9f704-3ecf-497a-8c90-39ff4c49f445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:22.648903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-41a9f704-3ecf-497a-8c90-39ff4c49f445 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:26:22.650090 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-41a9f704-3ecf-497a-8c90-39ff4c49f445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:26:22.721307 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 650/1297] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:26:22 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:26:22.748790 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a9869-f5dc-4236-adb9-9d23609c608a None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:22.847843 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a9869-f5dc-4236-adb9-9d23609c608a None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:22.881137 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d8dda-7620-4d69-94f5-d9227a74cd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:22.882236 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d8dda-7620-4d69-94f5-d9227a74cd14 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:22.882518 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d8dda-7620-4d69-94f5-d9227a74cd14 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:22.882795 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d8dda-7620-4d69-94f5-d9227a74cd14 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:22.883417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d8dda-7620-4d69-94f5-d9227a74cd14 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:22.884780 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d8dda-7620-4d69-94f5-d9227a74cd14 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:22.893892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-d76d8dda-7620-4d69-94f5-d9227a74cd14 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 16:26:22.893892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:22.893892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:26:22.893892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:26:22.893892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:26:22.893892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:26:22.893892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:26:22.893892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:26:22.893892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 16:26:22.893892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:26:22.893892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:26:22.893892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:26:22.893892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:22.893892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:22.893892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:22.893892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:22.893892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:22.894989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:22.894989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:22.894989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:22.894989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:22.894989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:22.894989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:22.894989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:26:22.894989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:26:22.894989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:26:22.894989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:26:22.894989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:22.894989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:22.894989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:22.894989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:22.894989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:22.894989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:22.895937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:22.895937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:22.895937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:22.895937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:22.895937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:22.895937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:22.895937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:22.895937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:22.895937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:22.895937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:22.895937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:26:22.895937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:26:22.895937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 16:26:22.895937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:26:22.895937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 648/1298] 10.222.0.74 () {56 vars in 1117 bytes} [Wed Jul 27 16:26:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:26:22.902641 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9f704-3ecf-497a-8c90-39ff4c49f445 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:22.925254 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:23.013328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9f704-3ecf-497a-8c90-39ff4c49f445 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:23.047166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a235b-c45d-44c8-a9cd-bd4573483b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:23.047803 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a235b-c45d-44c8-a9cd-bd4573483b83 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:23.048087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a235b-c45d-44c8-a9cd-bd4573483b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:23.048361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a235b-c45d-44c8-a9cd-bd4573483b83 None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:23.049063 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a235b-c45d-44c8-a9cd-bd4573483b83 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:23.049369 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a235b-c45d-44c8-a9cd-bd4573483b83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:23.050749 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a235b-c45d-44c8-a9cd-bd4573483b83 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:23.055852 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 651/1299] 10.222.0.74 () {56 vars in 1116 bytes} [Wed Jul 27 16:26:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:26:23.061805 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d8dda-7620-4d69-94f5-d9227a74cd14 None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:23.105906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:23.174753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d8dda-7620-4d69-94f5-d9227a74cd14 None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:23.207031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a842525-9ca2-4e19-b39c-43ccae94f8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:23.207634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a842525-9ca2-4e19-b39c-43ccae94f8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:23.207910 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a842525-9ca2-4e19-b39c-43ccae94f8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:23.208180 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a842525-9ca2-4e19-b39c-43ccae94f8e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:23.208935 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a842525-9ca2-4e19-b39c-43ccae94f8e1 None admin] RBAC: Adding query filter params (interface=public, service_id=3273ceb1935148179a10cc42e9a3eba9, region_id=RegionOne) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:23.209243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a842525-9ca2-4e19-b39c-43ccae94f8e1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:23.211244 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a842525-9ca2-4e19-b39c-43ccae94f8e1 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:23.216065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 649/1300] 10.222.0.74 () {56 vars in 1251 bytes} [Wed Jul 27 16:26:23 2022] GET /identity/v3/endpoints?service_id=3273ceb1935148179a10cc42e9a3eba9&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:23.221826 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a235b-c45d-44c8-a9cd-bd4573483b83 None admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:23.300409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:23.319566 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a235b-c45d-44c8-a9cd-bd4573483b83 None admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:23.352188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4ee2c-8fb6-4337-8e7d-f035cf5e1f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:23.352790 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4ee2c-8fb6-4337-8e7d-f035cf5e1f83 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:23.353098 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4ee2c-8fb6-4337-8e7d-f035cf5e1f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:23.353373 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4ee2c-8fb6-4337-8e7d-f035cf5e1f83 None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:23.354031 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4ee2c-8fb6-4337-8e7d-f035cf5e1f83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:23.355380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4ee2c-8fb6-4337-8e7d-f035cf5e1f83 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:23.362041 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 652/1301] 10.222.0.74 () {56 vars in 1087 bytes} [Wed Jul 27 16:26:23 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:26:25.365357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abaf9d-e39b-43bf-9f9d-f9168a4ecdd1 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:25.365973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abaf9d-e39b-43bf-9f9d-f9168a4ecdd1 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:25.365973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abaf9d-e39b-43bf-9f9d-f9168a4ecdd1 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:25.367318 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 650/1302] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:26:25.372691 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e319ff-c0ab-473d-938c-1a98d085af68 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:25.373154 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e319ff-c0ab-473d-938c-1a98d085af68 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:25.373279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e319ff-c0ab-473d-938c-1a98d085af68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:25.391711 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-43e319ff-c0ab-473d-938c-1a98d085af68 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:26:25.392924 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-43e319ff-c0ab-473d-938c-1a98d085af68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:26:25.458507 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 653/1303] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:26:25 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 16:26:25.480397 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abaf9d-e39b-43bf-9f9d-f9168a4ecdd1 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:25.573092 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:25.580188 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abaf9d-e39b-43bf-9f9d-f9168a4ecdd1 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:25.613146 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7612b9-fbc8-4741-a9cc-09ec07f95ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:25.613774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7612b9-fbc8-4741-a9cc-09ec07f95ccf None admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:25.614056 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7612b9-fbc8-4741-a9cc-09ec07f95ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:25.614328 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7612b9-fbc8-4741-a9cc-09ec07f95ccf None admin] PATH_INFO: `/v3/services/placement` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:25.614947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7612b9-fbc8-4741-a9cc-09ec07f95ccf None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:25.616337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7612b9-fbc8-4741-a9cc-09ec07f95ccf None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:25.624995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application [None req-3b7612b9-fbc8-4741-a9cc-09ec07f95ccf None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 16:26:25.624995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:25.624995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 16:26:25.624995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 16:26:25.624995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 16:26:25.624995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 16:26:25.624995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 16:26:25.624995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:26:25.624995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 16:26:25.624995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:26:25.624995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 16:26:25.624995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:26:25.624995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 16:26:25.624995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 16:26:25.624995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 16:26:25.624995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 16:26:25.624995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 16:26:25.626176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 16:26:25.626176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 16:26:25.626176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 16:26:25.626176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 16:26:25.626176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 16:26:25.626176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 16:26:25.626176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 16:26:25.626176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 16:26:25.626176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 16:26:25.626176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 16:26:25.626176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 16:26:25.626176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 16:26:25.626176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 16:26:25.626176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 16:26:25.626176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 16:26:25.626176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 16:26:25.627198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 16:26:25.627198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application with Lock( Jul 27 16:26:25.627198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 16:26:25.627198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self._enter() Jul 27 16:26:25.627198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 16:26:25.627198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 16:26:25.627198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 16:26:25.627198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application return self.creator() Jul 27 16:26:25.627198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 16:26:25.627198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application created_value = creator( Jul 27 16:26:25.627198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 16:26:25.627198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 16:26:25.627198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 16:26:25.627198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: ERROR keystone.server.flask.application Jul 27 16:26:25.627198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 651/1304] 10.222.0.74 () {56 vars in 1117 bytes} [Wed Jul 27 16:26:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 16:26:25.632033 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e319ff-c0ab-473d-938c-1a98d085af68 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:25.707006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:25.730636 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e319ff-c0ab-473d-938c-1a98d085af68 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:25.765345 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c578d7-06b5-4272-a9b3-c3204b000648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:25.765976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c578d7-06b5-4272-a9b3-c3204b000648 None admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:25.766261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c578d7-06b5-4272-a9b3-c3204b000648 None admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:25.766540 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c578d7-06b5-4272-a9b3-c3204b000648 None admin] PATH_INFO: `/v3/services` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:25.767226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c578d7-06b5-4272-a9b3-c3204b000648 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:26:25.767533 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c578d7-06b5-4272-a9b3-c3204b000648 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:25.768966 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c578d7-06b5-4272-a9b3-c3204b000648 None admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:25.777530 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 654/1305] 10.222.0.74 () {56 vars in 1116 bytes} [Wed Jul 27 16:26:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:26:25.783275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7612b9-fbc8-4741-a9cc-09ec07f95ccf None admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:26:25.885447 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7612b9-fbc8-4741-a9cc-09ec07f95ccf None admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:26:25.922063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52391dab-f178-4fcb-be20-0c774bc25028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:26:25.922686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52391dab-f178-4fcb-be20-0c774bc25028 None admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:26:25.922969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52391dab-f178-4fcb-be20-0c774bc25028 None admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:26:25.923243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52391dab-f178-4fcb-be20-0c774bc25028 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:26:25.923955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52391dab-f178-4fcb-be20-0c774bc25028 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:26:25.925368 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52391dab-f178-4fcb-be20-0c774bc25028 None admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:26:25.946982 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 652/1306] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:26:25 2022] POST /identity/v3/endpoints => generated 331 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:26:35.774344 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=30 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:26:35.942036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:27:28.392562 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb1aa4-c513-4062-8f2a-f0d8f7529eb1 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:28.393132 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb1aa4-c513-4062-8f2a-f0d8f7529eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:28.393132 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb1aa4-c513-4062-8f2a-f0d8f7529eb1 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:28.398883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 655/1307] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:27:28.400501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456068b-81c4-4640-8d58-1ba590592104 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:28.400865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456068b-81c4-4640-8d58-1ba590592104 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:28.401128 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456068b-81c4-4640-8d58-1ba590592104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:28.512400 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-d456068b-81c4-4640-8d58-1ba590592104 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:27:28.514126 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d456068b-81c4-4640-8d58-1ba590592104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:28.652664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 653/1308] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:27:28.707551 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86382b-4ee6-4690-8c25-ca9f33abcbfa None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:28.707830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86382b-4ee6-4690-8c25-ca9f33abcbfa None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:28.708139 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86382b-4ee6-4690-8c25-ca9f33abcbfa None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:28.710128 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 656/1309] 10.222.0.74 () {54 vars in 965 bytes} [Wed Jul 27 16:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:27:28.718845 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949ebea-e047-41ad-9ada-72784b3c415b None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:28.719130 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949ebea-e047-41ad-9ada-72784b3c415b None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:28.719403 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949ebea-e047-41ad-9ada-72784b3c415b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:28.798721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-1949ebea-e047-41ad-9ada-72784b3c415b None None] MFA Rules not processed for user `193dd9fee1844275ad0dab602e93df11`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:27:28.799927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-1949ebea-e047-41ad-9ada-72784b3c415b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:28.821964 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:27:28.887488 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 654/1310] 10.222.0.74 () {58 vars in 1062 bytes} [Wed Jul 27 16:27:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:27:28.897345 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b86382b-4ee6-4690-8c25-ca9f33abcbfa None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:27:28.899965 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-9b86382b-4ee6-4690-8c25-ca9f33abcbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:29.030500 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b86382b-4ee6-4690-8c25-ca9f33abcbfa None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:27:29.115611 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a026428-2068-482f-8f71-32ed425915d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:27:29.116234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a026428-2068-482f-8f71-32ed425915d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:29.116522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a026428-2068-482f-8f71-32ed425915d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:29.116798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a026428-2068-482f-8f71-32ed425915d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:29.118559 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-2a026428-2068-482f-8f71-32ed425915d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:29.182689 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:27:29.244489 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a026428-2068-482f-8f71-32ed425915d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:27:29.245967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a026428-2068-482f-8f71-32ed425915d8 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:27:29.268595 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 657/1311] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:27:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:27:31.543144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672ccc9-9fae-487f-b116-cbaf491a5157 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:31.544026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672ccc9-9fae-487f-b116-cbaf491a5157 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:31.544307 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672ccc9-9fae-487f-b116-cbaf491a5157 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:31.545877 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 655/1312] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:27:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:27:31.550786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5783b-3b27-4a24-8938-f365f3213908 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:31.551214 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5783b-3b27-4a24-8938-f365f3213908 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:31.551443 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5783b-3b27-4a24-8938-f365f3213908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:31.647765 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-c9e5783b-3b27-4a24-8938-f365f3213908 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:27:31.649423 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-c9e5783b-3b27-4a24-8938-f365f3213908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:31.764999 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 658/1313] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:27:31.803999 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672ccc9-9fae-487f-b116-cbaf491a5157 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:27:31.805952 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-4672ccc9-9fae-487f-b116-cbaf491a5157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:31.955429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672ccc9-9fae-487f-b116-cbaf491a5157 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:27:32.035805 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:27:32.039156 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8210517-424f-4348-b6cc-e15089d568fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:27:32.039771 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8210517-424f-4348-b6cc-e15089d568fd service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:32.040086 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8210517-424f-4348-b6cc-e15089d568fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:32.040436 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8210517-424f-4348-b6cc-e15089d568fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:32.042554 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d8210517-424f-4348-b6cc-e15089d568fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:32.165007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8210517-424f-4348-b6cc-e15089d568fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:27:32.165007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8210517-424f-4348-b6cc-e15089d568fd service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:27:32.186479 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 656/1314] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:27:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:27:34.473024 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505510be-1a81-44ec-9559-d063d32d8b36 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:34.474036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505510be-1a81-44ec-9559-d063d32d8b36 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:34.474132 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505510be-1a81-44ec-9559-d063d32d8b36 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:34.475481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 659/1315] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:27:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:27:34.480617 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a21128-e1e2-4e14-b5eb-fd56c335ae29 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:34.481069 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a21128-e1e2-4e14-b5eb-fd56c335ae29 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:34.481189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a21128-e1e2-4e14-b5eb-fd56c335ae29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:34.546104 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-21a21128-e1e2-4e14-b5eb-fd56c335ae29 None None] MFA Rules not processed for user `dcba272d6ea54652b4d3c43cf492f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:27:34.547321 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-21a21128-e1e2-4e14-b5eb-fd56c335ae29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:34.597518 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:27:34.642053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 657/1316] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:27:34 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:27:34.697655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba910d1-c70f-405d-816e-0f1bf692dbfb None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:34.697939 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba910d1-c70f-405d-816e-0f1bf692dbfb None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:34.698215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba910d1-c70f-405d-816e-0f1bf692dbfb None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:34.699558 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 660/1317] 10.222.0.74 () {54 vars in 965 bytes} [Wed Jul 27 16:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:27:34.705579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525263c9-e8a7-448f-afdf-ba4d7a489048 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:34.705877 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525263c9-e8a7-448f-afdf-ba4d7a489048 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:34.706161 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525263c9-e8a7-448f-afdf-ba4d7a489048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:34.780688 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-525263c9-e8a7-448f-afdf-ba4d7a489048 None None] MFA Rules not processed for user `193dd9fee1844275ad0dab602e93df11`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:27:34.781866 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-525263c9-e8a7-448f-afdf-ba4d7a489048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:34.856985 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 658/1318] 10.222.0.74 () {58 vars in 1062 bytes} [Wed Jul 27 16:27:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:27:34.865997 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba910d1-c70f-405d-816e-0f1bf692dbfb None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:27:34.867526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-fba910d1-c70f-405d-816e-0f1bf692dbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:34.877115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:27:34.969817 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba910d1-c70f-405d-816e-0f1bf692dbfb None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:27:35.041076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9c3a4-0371-4111-a350-65e781442f6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:27:35.041625 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9c3a4-0371-4111-a350-65e781442f6a service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:35.041909 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9c3a4-0371-4111-a350-65e781442f6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:35.042186 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9c3a4-0371-4111-a350-65e781442f6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:35.043813 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e2e9c3a4-0371-4111-a350-65e781442f6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:35.106010 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:27:35.157043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9c3a4-0371-4111-a350-65e781442f6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:27:35.158492 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9c3a4-0371-4111-a350-65e781442f6a service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:27:35.176382 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 661/1319] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:27:34 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:27:38.137386 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f08b52-9fea-42c1-99d2-d8428d3147f5 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:38.137921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f08b52-9fea-42c1-99d2-d8428d3147f5 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:38.137921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f08b52-9fea-42c1-99d2-d8428d3147f5 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:38.139282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 659/1320] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:27:38.144333 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e16e84-ad60-49eb-a4e0-7a6935aaaac5 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:38.144761 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e16e84-ad60-49eb-a4e0-7a6935aaaac5 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:38.144916 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e16e84-ad60-49eb-a4e0-7a6935aaaac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:38.210648 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-84e16e84-ad60-49eb-a4e0-7a6935aaaac5 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:27:38.211866 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-84e16e84-ad60-49eb-a4e0-7a6935aaaac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:38.307010 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 662/1321] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:27:38 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:27:38.346160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f08b52-9fea-42c1-99d2-d8428d3147f5 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:27:38.347686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-99f08b52-9fea-42c1-99d2-d8428d3147f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:38.397846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:27:38.451143 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f08b52-9fea-42c1-99d2-d8428d3147f5 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:27:38.524299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635fa928-5f7c-45c7-9438-989b9c2b4d06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:27:38.524936 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fa928-5f7c-45c7-9438-989b9c2b4d06 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:38.525224 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fa928-5f7c-45c7-9438-989b9c2b4d06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:38.525503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fa928-5f7c-45c7-9438-989b9c2b4d06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:38.527112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-635fa928-5f7c-45c7-9438-989b9c2b4d06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:38.621849 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:27:38.635913 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635fa928-5f7c-45c7-9438-989b9c2b4d06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:27:38.637390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635fa928-5f7c-45c7-9438-989b9c2b4d06 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:27:38.656041 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 660/1322] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:27:38 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:27:41.431238 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926e229-b937-4f97-923f-d186a8d42fe6 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:41.431740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926e229-b937-4f97-923f-d186a8d42fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:41.431998 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926e229-b937-4f97-923f-d186a8d42fe6 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:41.434180 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 663/1323] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:27:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:27:41.440453 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbbdd7-bc95-4a7a-9044-b82de04fafc1 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:41.440869 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbbdd7-bc95-4a7a-9044-b82de04fafc1 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:41.441081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbbdd7-bc95-4a7a-9044-b82de04fafc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:41.506832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-02fbbdd7-bc95-4a7a-9044-b82de04fafc1 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:27:41.508037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-02fbbdd7-bc95-4a7a-9044-b82de04fafc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:41.611349 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 661/1324] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:27:41.648734 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926e229-b937-4f97-923f-d186a8d42fe6 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:27:41.650448 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e926e229-b937-4f97-923f-d186a8d42fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:41.671402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:27:41.774103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926e229-b937-4f97-923f-d186a8d42fe6 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:27:41.849399 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ac664-3c39-4bab-a33d-e97c9ba02ef0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:27:41.850016 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ac664-3c39-4bab-a33d-e97c9ba02ef0 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:41.850296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ac664-3c39-4bab-a33d-e97c9ba02ef0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:41.850568 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ac664-3c39-4bab-a33d-e97c9ba02ef0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:41.852189 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-6b4ac664-3c39-4bab-a33d-e97c9ba02ef0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:41.923751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:27:41.965035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ac664-3c39-4bab-a33d-e97c9ba02ef0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:27:41.966483 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ac664-3c39-4bab-a33d-e97c9ba02ef0 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:27:41.986248 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 664/1325] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:27:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:27:44.058078 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54707ab-0b24-414d-9e2b-75dc87e8c705 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:44.058725 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54707ab-0b24-414d-9e2b-75dc87e8c705 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:44.058725 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54707ab-0b24-414d-9e2b-75dc87e8c705 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:44.060286 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 662/1326] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:27:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:27:44.065466 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587b68d-5396-4191-931c-f8ef6a3da5f0 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:44.065898 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587b68d-5396-4191-931c-f8ef6a3da5f0 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:44.066052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587b68d-5396-4191-931c-f8ef6a3da5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:44.131954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-d587b68d-5396-4191-931c-f8ef6a3da5f0 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:27:44.133171 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d587b68d-5396-4191-931c-f8ef6a3da5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:44.232862 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 665/1327] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:27:44.268941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54707ab-0b24-414d-9e2b-75dc87e8c705 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:27:44.271019 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-e54707ab-0b24-414d-9e2b-75dc87e8c705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:44.329919 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:27:44.373663 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54707ab-0b24-414d-9e2b-75dc87e8c705 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:27:44.451728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4b1a3-46af-4f0e-ba08-9649269c3d21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:27:44.452347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4b1a3-46af-4f0e-ba08-9649269c3d21 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:44.452631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4b1a3-46af-4f0e-ba08-9649269c3d21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:44.452930 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4b1a3-46af-4f0e-ba08-9649269c3d21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:44.454812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-81f4b1a3-46af-4f0e-ba08-9649269c3d21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:44.561896 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:27:44.564456 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4b1a3-46af-4f0e-ba08-9649269c3d21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:27:44.565939 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4b1a3-46af-4f0e-ba08-9649269c3d21 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:27:44.584954 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 663/1328] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:27:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:27:46.690719 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eca2e3-6e12-4e87-be80-3e04d239b05f None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:46.691243 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eca2e3-6e12-4e87-be80-3e04d239b05f None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:46.691355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eca2e3-6e12-4e87-be80-3e04d239b05f None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:46.693448 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 666/1329] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:27:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:27:46.698460 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d14157-f3b8-4719-a97a-c4a772096f6a None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:46.698884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d14157-f3b8-4719-a97a-c4a772096f6a None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:46.698998 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d14157-f3b8-4719-a97a-c4a772096f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:46.767027 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-03d14157-f3b8-4719-a97a-c4a772096f6a None None] MFA Rules not processed for user `dcba272d6ea54652b4d3c43cf492f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:27:46.768260 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-03d14157-f3b8-4719-a97a-c4a772096f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:46.868391 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 664/1330] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:27:46 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:27:46.905960 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eca2e3-6e12-4e87-be80-3e04d239b05f None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:27:46.907562 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-c5eca2e3-6e12-4e87-be80-3e04d239b05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:46.938932 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:27:47.012666 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eca2e3-6e12-4e87-be80-3e04d239b05f None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:27:47.087874 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac185093-f482-41fd-bd10-365fdea9e5fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:27:47.088493 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac185093-f482-41fd-bd10-365fdea9e5fd service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:47.088777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac185093-f482-41fd-bd10-365fdea9e5fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:47.089085 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac185093-f482-41fd-bd10-365fdea9e5fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:47.090720 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-ac185093-f482-41fd-bd10-365fdea9e5fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:47.169727 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:27:47.204131 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac185093-f482-41fd-bd10-365fdea9e5fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:27:47.205751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac185093-f482-41fd-bd10-365fdea9e5fd service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:27:47.244655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 667/1331] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:27:46 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:27:49.518284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6d8df-47e4-454b-8782-912536f04b4e None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:49.518842 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6d8df-47e4-454b-8782-912536f04b4e None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:49.518941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6d8df-47e4-454b-8782-912536f04b4e None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:49.520782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 665/1332] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:27:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:27:49.526050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29daa359-0722-404c-bfc6-d4005d14d853 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:49.526489 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29daa359-0722-404c-bfc6-d4005d14d853 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:49.526598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29daa359-0722-404c-bfc6-d4005d14d853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:49.591701 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-29daa359-0722-404c-bfc6-d4005d14d853 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:27:49.592816 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-29daa359-0722-404c-bfc6-d4005d14d853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:49.686883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 668/1333] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:27:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:27:49.722991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6d8df-47e4-454b-8782-912536f04b4e None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:27:49.725144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-e3d6d8df-47e4-454b-8782-912536f04b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:49.792216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:27:49.831853 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6d8df-47e4-454b-8782-912536f04b4e None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:27:49.905687 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d26442-ef7f-43ab-a5f2-2f120c60c3f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:27:49.906243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d26442-ef7f-43ab-a5f2-2f120c60c3f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:49.906527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d26442-ef7f-43ab-a5f2-2f120c60c3f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:49.906806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d26442-ef7f-43ab-a5f2-2f120c60c3f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:49.908424 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-f3d26442-ef7f-43ab-a5f2-2f120c60c3f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:50.015387 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d26442-ef7f-43ab-a5f2-2f120c60c3f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:27:50.016833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d26442-ef7f-43ab-a5f2-2f120c60c3f0 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:27:50.026760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:27:50.035246 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 666/1334] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:27:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:27:52.086502 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e810ce4-9f4c-4b3b-a506-e7203b095c5e None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:52.087034 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e810ce4-9f4c-4b3b-a506-e7203b095c5e None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:52.087034 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e810ce4-9f4c-4b3b-a506-e7203b095c5e None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:52.088408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 669/1335] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:27:52.093504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ad547-e2e7-4327-8165-491a19f1686d None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:52.093890 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ad547-e2e7-4327-8165-491a19f1686d None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:52.094062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ad547-e2e7-4327-8165-491a19f1686d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:52.160406 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-d97ad547-e2e7-4327-8165-491a19f1686d None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:27:52.162145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d97ad547-e2e7-4327-8165-491a19f1686d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:52.258427 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 667/1336] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:27:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:27:52.299724 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e810ce4-9f4c-4b3b-a506-e7203b095c5e None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:27:52.301905 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-6e810ce4-9f4c-4b3b-a506-e7203b095c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:52.341115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:27:52.404691 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e810ce4-9f4c-4b3b-a506-e7203b095c5e None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:27:52.477612 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94635e19-7d05-4316-82bc-f6e10c43c485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:27:52.478231 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94635e19-7d05-4316-82bc-f6e10c43c485 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:52.478513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94635e19-7d05-4316-82bc-f6e10c43c485 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:52.478789 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94635e19-7d05-4316-82bc-f6e10c43c485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:52.480410 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-94635e19-7d05-4316-82bc-f6e10c43c485 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:52.562459 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:27:52.588608 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94635e19-7d05-4316-82bc-f6e10c43c485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:27:52.590088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94635e19-7d05-4316-82bc-f6e10c43c485 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:27:52.608392 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 670/1337] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:27:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:27:55.033226 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23531ee6-56da-4f84-8bac-e9f3c15b18fe None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:55.033824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23531ee6-56da-4f84-8bac-e9f3c15b18fe None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:55.033824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23531ee6-56da-4f84-8bac-e9f3c15b18fe None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:55.035118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 668/1338] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:27:55.040795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a75733-bb65-490b-a000-1501abfc0723 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:55.041319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a75733-bb65-490b-a000-1501abfc0723 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:55.041426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a75733-bb65-490b-a000-1501abfc0723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:55.106433 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-60a75733-bb65-490b-a000-1501abfc0723 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:27:55.107630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-60a75733-bb65-490b-a000-1501abfc0723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:55.205952 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 671/1339] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:27:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:27:55.242808 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23531ee6-56da-4f84-8bac-e9f3c15b18fe None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:27:55.244348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-23531ee6-56da-4f84-8bac-e9f3c15b18fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:55.321896 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:27:55.350250 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23531ee6-56da-4f84-8bac-e9f3c15b18fe None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:27:55.421622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4769f0b-2939-4808-9d5e-3769b35866f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:27:55.422241 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4769f0b-2939-4808-9d5e-3769b35866f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:55.422519 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4769f0b-2939-4808-9d5e-3769b35866f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:55.422793 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4769f0b-2939-4808-9d5e-3769b35866f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:55.424403 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-e4769f0b-2939-4808-9d5e-3769b35866f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:55.530748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4769f0b-2939-4808-9d5e-3769b35866f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:27:55.532195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4769f0b-2939-4808-9d5e-3769b35866f1 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:27:55.549652 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 669/1340] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:27:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:27:59.486048 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c7a9a-5f9c-4183-848c-11762c63a4c8 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:59.486560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c7a9a-5f9c-4183-848c-11762c63a4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:59.486658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c7a9a-5f9c-4183-848c-11762c63a4c8 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:59.488010 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 672/1341] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:27:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:27:59.493802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a08c9-35dc-4d31-ab32-6bf5096db8c8 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:59.494189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a08c9-35dc-4d31-ab32-6bf5096db8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:59.494484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a08c9-35dc-4d31-ab32-6bf5096db8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:59.500060 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:27:59.562924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-c38a08c9-35dc-4d31-ab32-6bf5096db8c8 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:27:59.564153 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c38a08c9-35dc-4d31-ab32-6bf5096db8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:59.665492 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 670/1342] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:27:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:27:59.704420 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c7a9a-5f9c-4183-848c-11762c63a4c8 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:27:59.705938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-5b0c7a9a-5f9c-4183-848c-11762c63a4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:59.753650 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:27:59.810921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c7a9a-5f9c-4183-848c-11762c63a4c8 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:27:59.885980 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8944ab-d48a-42ab-8b08-3c28e7d342d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:27:59.886603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8944ab-d48a-42ab-8b08-3c28e7d342d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:27:59.886879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8944ab-d48a-42ab-8b08-3c28e7d342d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:27:59.887153 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8944ab-d48a-42ab-8b08-3c28e7d342d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:27:59.888773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-5b8944ab-d48a-42ab-8b08-3c28e7d342d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:27:59.983081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:00.000533 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8944ab-d48a-42ab-8b08-3c28e7d342d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:28:00.002011 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8944ab-d48a-42ab-8b08-3c28e7d342d1 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:28:00.021456 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 673/1343] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:27:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:28:02.682279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f76490-e611-4a53-aca0-5514ee70b644 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:02.682855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f76490-e611-4a53-aca0-5514ee70b644 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:02.682855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f76490-e611-4a53-aca0-5514ee70b644 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:02.684401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 671/1344] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:28:02.690090 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac74440-63b1-40b3-8df3-2cd020ff13da None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:02.690670 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac74440-63b1-40b3-8df3-2cd020ff13da None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:02.690670 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac74440-63b1-40b3-8df3-2cd020ff13da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:02.767749 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-0ac74440-63b1-40b3-8df3-2cd020ff13da None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:02.769017 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-0ac74440-63b1-40b3-8df3-2cd020ff13da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:02.873156 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 674/1345] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:28:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:28:02.913875 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f76490-e611-4a53-aca0-5514ee70b644 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:28:02.916562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-a7f76490-e611-4a53-aca0-5514ee70b644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:03.013289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:03.030774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f76490-e611-4a53-aca0-5514ee70b644 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:28:03.103519 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075ec34-c13b-4cae-98e9-6031e5b90395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:28:03.104050 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075ec34-c13b-4cae-98e9-6031e5b90395 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:03.104323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075ec34-c13b-4cae-98e9-6031e5b90395 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:03.104593 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075ec34-c13b-4cae-98e9-6031e5b90395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:03.106223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c075ec34-c13b-4cae-98e9-6031e5b90395 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:03.214368 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075ec34-c13b-4cae-98e9-6031e5b90395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:28:03.215829 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075ec34-c13b-4cae-98e9-6031e5b90395 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:28:03.233444 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 672/1346] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:28:02 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:28:07.193068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e88c80-e075-4c82-96db-ff2f7727c21b None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:07.193605 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e88c80-e075-4c82-96db-ff2f7727c21b None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:07.193605 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e88c80-e075-4c82-96db-ff2f7727c21b None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:07.195130 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 675/1347] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:28:07.200104 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd982322-d5bc-4f4d-b505-dc9969d7adba None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:07.200530 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd982322-d5bc-4f4d-b505-dc9969d7adba None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:07.200657 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd982322-d5bc-4f4d-b505-dc9969d7adba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:07.216398 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:07.268054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-bd982322-d5bc-4f4d-b505-dc9969d7adba None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:07.269328 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-bd982322-d5bc-4f4d-b505-dc9969d7adba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:07.370977 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 673/1348] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:28:07.414362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e88c80-e075-4c82-96db-ff2f7727c21b None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:28:07.415887 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-97e88c80-e075-4c82-96db-ff2f7727c21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:07.482272 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:07.533643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e88c80-e075-4c82-96db-ff2f7727c21b None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:28:07.611249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893d27e-c78a-4183-8e9b-9f4b823703c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:28:07.611882 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893d27e-c78a-4183-8e9b-9f4b823703c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:07.612160 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893d27e-c78a-4183-8e9b-9f4b823703c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:07.612431 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893d27e-c78a-4183-8e9b-9f4b823703c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:07.614141 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-8893d27e-c78a-4183-8e9b-9f4b823703c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:07.727021 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:07.730748 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893d27e-c78a-4183-8e9b-9f4b823703c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:28:07.732509 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893d27e-c78a-4183-8e9b-9f4b823703c7 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:28:07.751567 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 676/1349] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:28:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:28:12.750690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505e33d-0ba9-4cf3-be6c-e9bbd1a2597a None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:12.751666 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505e33d-0ba9-4cf3-be6c-e9bbd1a2597a None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:12.752164 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505e33d-0ba9-4cf3-be6c-e9bbd1a2597a None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:12.753977 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 674/1350] 10.222.0.74 () {54 vars in 921 bytes} [Wed Jul 27 16:28:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:28:12.763004 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768fd93-fceb-4b7f-b7dd-e630af2ac3db None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:12.763920 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768fd93-fceb-4b7f-b7dd-e630af2ac3db None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:12.764422 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768fd93-fceb-4b7f-b7dd-e630af2ac3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:12.870732 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-9768fd93-fceb-4b7f-b7dd-e630af2ac3db None None] MFA Rules not processed for user `91f8e0f82a8541aead6cd8cc45f4a86b`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:12.871852 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-9768fd93-fceb-4b7f-b7dd-e630af2ac3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:13.002032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 677/1351] 10.222.0.74 () {58 vars in 1018 bytes} [Wed Jul 27 16:28:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:28:17.056220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b09a28-685e-49d0-9997-fd03e262cd51 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:17.057132 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b09a28-685e-49d0-9997-fd03e262cd51 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:17.057573 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b09a28-685e-49d0-9997-fd03e262cd51 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:17.059383 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 675/1352] 10.222.0.74 () {54 vars in 921 bytes} [Wed Jul 27 16:28:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:28:17.064873 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229fed6-7dcf-489a-933b-c73d89682517 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:17.065848 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229fed6-7dcf-489a-933b-c73d89682517 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:17.066877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229fed6-7dcf-489a-933b-c73d89682517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:17.126198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:17.153240 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-d229fed6-7dcf-489a-933b-c73d89682517 None None] MFA Rules not processed for user `91f8e0f82a8541aead6cd8cc45f4a86b`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:17.154454 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d229fed6-7dcf-489a-933b-c73d89682517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:17.275879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 678/1353] 10.222.0.74 () {58 vars in 1018 bytes} [Wed Jul 27 16:28:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:28:17.368995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=59 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:18.640315 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77771c5-01cf-4647-adbe-3a7c573950c0 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:18.640978 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77771c5-01cf-4647-adbe-3a7c573950c0 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:18.640978 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77771c5-01cf-4647-adbe-3a7c573950c0 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:18.642399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 676/1354] 10.222.0.74 () {54 vars in 921 bytes} [Wed Jul 27 16:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:28:18.647265 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a4f45-7d49-45fe-b903-ae9f7695ad42 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:18.647712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a4f45-7d49-45fe-b903-ae9f7695ad42 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:18.647816 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a4f45-7d49-45fe-b903-ae9f7695ad42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:18.724269 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-013a4f45-7d49-45fe-b903-ae9f7695ad42 None None] MFA Rules not processed for user `91f8e0f82a8541aead6cd8cc45f4a86b`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:18.725550 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-013a4f45-7d49-45fe-b903-ae9f7695ad42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:18.766766 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:18.833706 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 679/1355] 10.222.0.74 () {58 vars in 1018 bytes} [Wed Jul 27 16:28:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:28:22.260610 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3b62c-1e0d-4134-bb9e-e9d78bab0a6f None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:22.261483 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3b62c-1e0d-4134-bb9e-e9d78bab0a6f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:22.261950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3b62c-1e0d-4134-bb9e-e9d78bab0a6f None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:22.267870 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 677/1356] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:28:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:28:22.271313 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01c8f0-5c81-4642-92b1-dfa827e98bdc None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:22.272091 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01c8f0-5c81-4642-92b1-dfa827e98bdc None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:22.272524 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01c8f0-5c81-4642-92b1-dfa827e98bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:22.356073 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-0b01c8f0-5c81-4642-92b1-dfa827e98bdc None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:22.357921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-0b01c8f0-5c81-4642-92b1-dfa827e98bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:22.468164 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 680/1357] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:28:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:28:22.521275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba767b0-5732-4896-b79b-dc4918a61efb None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:22.521809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba767b0-5732-4896-b79b-dc4918a61efb None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:22.522342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba767b0-5732-4896-b79b-dc4918a61efb None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:22.524187 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 678/1358] 10.222.0.74 () {54 vars in 962 bytes} [Wed Jul 27 16:28:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:28:22.544055 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3554089-5435-4277-bb7d-298d64abe443 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:22.544710 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3554089-5435-4277-bb7d-298d64abe443 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:22.545216 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3554089-5435-4277-bb7d-298d64abe443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:22.565043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:22.638782 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-f3554089-5435-4277-bb7d-298d64abe443 None None] MFA Rules not processed for user `56dfafd15e284b7986ba07dd3819bf65`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:22.640271 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f3554089-5435-4277-bb7d-298d64abe443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:22.770051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 681/1359] 10.222.0.74 () {58 vars in 1059 bytes} [Wed Jul 27 16:28:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:28:22.777631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba767b0-5732-4896-b79b-dc4918a61efb None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:28:22.779295 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6ba767b0-5732-4896-b79b-dc4918a61efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:22.947252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba767b0-5732-4896-b79b-dc4918a61efb None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:28:23.024463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:23.074415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89acde4b-8fb2-4d4b-8c65-347d3b84083e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:28:23.075635 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acde4b-8fb2-4d4b-8c65-347d3b84083e service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:23.076227 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acde4b-8fb2-4d4b-8c65-347d3b84083e service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:23.076741 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acde4b-8fb2-4d4b-8c65-347d3b84083e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:23.078592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-89acde4b-8fb2-4d4b-8c65-347d3b84083e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:23.183249 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89acde4b-8fb2-4d4b-8c65-347d3b84083e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:28:23.184843 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89acde4b-8fb2-4d4b-8c65-347d3b84083e service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:28:23.205622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 679/1360] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:28:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:28:25.430081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9f8f3-60b6-4061-a6ac-60b0fecfdc09 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:25.430581 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9f8f3-60b6-4061-a6ac-60b0fecfdc09 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:25.430581 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9f8f3-60b6-4061-a6ac-60b0fecfdc09 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:25.432055 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 682/1361] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:28:25.437210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb71aa-d59b-4b3b-8fe9-b101d23c4c03 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:25.437617 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb71aa-d59b-4b3b-8fe9-b101d23c4c03 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:25.437765 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb71aa-d59b-4b3b-8fe9-b101d23c4c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:25.508407 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-8cdb71aa-d59b-4b3b-8fe9-b101d23c4c03 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:25.509644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-8cdb71aa-d59b-4b3b-8fe9-b101d23c4c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:25.550499 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:25.628233 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 680/1362] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:28:25.683292 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9f8f3-60b6-4061-a6ac-60b0fecfdc09 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:28:25.685014 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-30d9f8f3-60b6-4061-a6ac-60b0fecfdc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:25.756135 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:25.848244 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9f8f3-60b6-4061-a6ac-60b0fecfdc09 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:28:25.972391 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e877a2-f77c-4cd3-a876-54a1fbde9ea2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:28:25.973054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e877a2-f77c-4cd3-a876-54a1fbde9ea2 service nova] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:25.973338 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e877a2-f77c-4cd3-a876-54a1fbde9ea2 service nova] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:25.973611 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e877a2-f77c-4cd3-a876-54a1fbde9ea2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:25.975241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-27e877a2-f77c-4cd3-a876-54a1fbde9ea2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:26.001331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:26.072228 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e877a2-f77c-4cd3-a876-54a1fbde9ea2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:28:26.073881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e877a2-f77c-4cd3-a876-54a1fbde9ea2 service nova] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:28:26.093930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 683/1363] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:28:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:28:28.485559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a60768-3fc4-4657-a89c-a1a2bef05fea None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:28.486221 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a60768-3fc4-4657-a89c-a1a2bef05fea None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:28.486221 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a60768-3fc4-4657-a89c-a1a2bef05fea None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:28.487502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 681/1364] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:28:28.494519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d11e47-542b-4e9f-9b40-58a817e19085 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:28.495012 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d11e47-542b-4e9f-9b40-58a817e19085 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:28.495128 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d11e47-542b-4e9f-9b40-58a817e19085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:28.575031 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-c4d11e47-542b-4e9f-9b40-58a817e19085 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:28.576159 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-c4d11e47-542b-4e9f-9b40-58a817e19085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:28.679081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:28.688941 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 684/1365] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:28:28.729772 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f25309-d8d0-4af2-903b-fb17afdbb531 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:28.730319 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f25309-d8d0-4af2-903b-fb17afdbb531 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:28.730665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f25309-d8d0-4af2-903b-fb17afdbb531 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:28.732151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 682/1366] 10.222.0.74 () {54 vars in 962 bytes} [Wed Jul 27 16:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:28:28.738672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b22ba-76e8-42bc-9767-764daa117cbe None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:28.739027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b22ba-76e8-42bc-9767-764daa117cbe None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:28.739367 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b22ba-76e8-42bc-9767-764daa117cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:28.832630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-165b22ba-76e8-42bc-9767-764daa117cbe None None] MFA Rules not processed for user `56dfafd15e284b7986ba07dd3819bf65`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:28.833853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-165b22ba-76e8-42bc-9767-764daa117cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:28.959223 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 685/1367] 10.222.0.74 () {58 vars in 1059 bytes} [Wed Jul 27 16:28:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:28:28.967267 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f25309-d8d0-4af2-903b-fb17afdbb531 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:28:28.968923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-e7f25309-d8d0-4af2-903b-fb17afdbb531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:29.134355 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f25309-d8d0-4af2-903b-fb17afdbb531 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:28:29.159495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:29.279487 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b3c79-57be-44fe-a3ca-e64d4f914f79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:28:29.280138 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b3c79-57be-44fe-a3ca-e64d4f914f79 service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:29.280415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b3c79-57be-44fe-a3ca-e64d4f914f79 service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:29.280684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b3c79-57be-44fe-a3ca-e64d4f914f79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:29.282310 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9a6b3c79-57be-44fe-a3ca-e64d4f914f79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:29.376059 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b3c79-57be-44fe-a3ca-e64d4f914f79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:28:29.377651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b3c79-57be-44fe-a3ca-e64d4f914f79 service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:28:29.397494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 683/1368] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:28:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:28:30.369053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cec0f-d91a-4691-b24d-c266fedd00cc None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:30.369618 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cec0f-d91a-4691-b24d-c266fedd00cc None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:30.369618 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cec0f-d91a-4691-b24d-c266fedd00cc None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:30.371120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 686/1369] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:28:30.376086 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3ce05-4a0d-48d0-ba66-290ed8ddd103 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:30.376533 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3ce05-4a0d-48d0-ba66-290ed8ddd103 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:30.376635 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3ce05-4a0d-48d0-ba66-290ed8ddd103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:30.410184 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:30.448601 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-6eb3ce05-4a0d-48d0-ba66-290ed8ddd103 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:30.449812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6eb3ce05-4a0d-48d0-ba66-290ed8ddd103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:30.554633 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 684/1370] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:28:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:28:30.866696 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419014b-7199-4267-9cfb-027ddb174a33 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:30.867069 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419014b-7199-4267-9cfb-027ddb174a33 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:30.867534 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419014b-7199-4267-9cfb-027ddb174a33 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:30.869079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 687/1371] 10.222.0.74 () {54 vars in 964 bytes} [Wed Jul 27 16:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:28:30.874057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba73680-f578-4e86-b5a8-2054940eda7e None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:30.874350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba73680-f578-4e86-b5a8-2054940eda7e None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:30.874628 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba73680-f578-4e86-b5a8-2054940eda7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:30.955697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-5ba73680-f578-4e86-b5a8-2054940eda7e None None] MFA Rules not processed for user `dd352d423bff418f932b434c51b15d23`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:30.957723 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-5ba73680-f578-4e86-b5a8-2054940eda7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:31.010494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:31.043911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 685/1372] 10.222.0.74 () {58 vars in 1061 bytes} [Wed Jul 27 16:28:30 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:28:31.052520 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419014b-7199-4267-9cfb-027ddb174a33 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:28:31.054085 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f419014b-7199-4267-9cfb-027ddb174a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:31.108525 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:31.176002 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419014b-7199-4267-9cfb-027ddb174a33 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:28:31.253214 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09d503-2fda-4818-b216-5addd6270a33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:28:31.254363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09d503-2fda-4818-b216-5addd6270a33 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:31.254648 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09d503-2fda-4818-b216-5addd6270a33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:31.254921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09d503-2fda-4818-b216-5addd6270a33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:31.256554 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7b09d503-2fda-4818-b216-5addd6270a33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:31.361277 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:31.374512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09d503-2fda-4818-b216-5addd6270a33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:28:31.376152 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09d503-2fda-4818-b216-5addd6270a33 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:28:31.397501 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 688/1373] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:28:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:28:31.485047 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c839f-bd27-400b-b6f3-34e0db7636d3 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:31.485501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c839f-bd27-400b-b6f3-34e0db7636d3 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:31.485739 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c839f-bd27-400b-b6f3-34e0db7636d3 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:31.487094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 686/1374] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:28:31.492680 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4040767-a8e1-4cf9-b2bd-5574a7130ef6 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:31.493189 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4040767-a8e1-4cf9-b2bd-5574a7130ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:31.493337 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4040767-a8e1-4cf9-b2bd-5574a7130ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:31.569952 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-d4040767-a8e1-4cf9-b2bd-5574a7130ef6 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:31.571464 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d4040767-a8e1-4cf9-b2bd-5574a7130ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:31.689390 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 689/1375] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:28:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:28:31.725453 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c839f-bd27-400b-b6f3-34e0db7636d3 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:28:31.725832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-602c839f-bd27-400b-b6f3-34e0db7636d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:31.890938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c839f-bd27-400b-b6f3-34e0db7636d3 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:28:31.942491 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:32.003052 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ef9a0-e96d-4a86-bcea-1edbbed8bbd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:28:32.003698 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ef9a0-e96d-4a86-bcea-1edbbed8bbd2 service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:32.003978 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ef9a0-e96d-4a86-bcea-1edbbed8bbd2 service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:32.004254 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ef9a0-e96d-4a86-bcea-1edbbed8bbd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:32.005908 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-5e3ef9a0-e96d-4a86-bcea-1edbbed8bbd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:32.096495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ef9a0-e96d-4a86-bcea-1edbbed8bbd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:28:32.098793 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ef9a0-e96d-4a86-bcea-1edbbed8bbd2 service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:28:32.117738 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 687/1376] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:28:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:28:34.210747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2769a1c-3478-4cd1-8067-a133e6b567b6 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:34.211301 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2769a1c-3478-4cd1-8067-a133e6b567b6 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:34.211301 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2769a1c-3478-4cd1-8067-a133e6b567b6 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:34.212672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 690/1377] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:28:34.217772 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7e7ac-7515-4d2c-b13a-3fd747f90118 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:34.218200 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7e7ac-7515-4d2c-b13a-3fd747f90118 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:34.218329 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7e7ac-7515-4d2c-b13a-3fd747f90118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:34.294837 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-a3f7e7ac-7515-4d2c-b13a-3fd747f90118 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:34.296408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-a3f7e7ac-7515-4d2c-b13a-3fd747f90118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:34.303307 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:34.401674 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 688/1378] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:28:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:28:34.461354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2769a1c-3478-4cd1-8067-a133e6b567b6 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:28:34.462906 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-a2769a1c-3478-4cd1-8067-a133e6b567b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:34.524651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:34.614439 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2769a1c-3478-4cd1-8067-a133e6b567b6 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:28:34.732539 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f341ce-c53d-4011-9234-d7c4626a2452 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:28:34.733187 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f341ce-c53d-4011-9234-d7c4626a2452 service nova] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:34.733470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f341ce-c53d-4011-9234-d7c4626a2452 service nova] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:34.733747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f341ce-c53d-4011-9234-d7c4626a2452 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:34.735357 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-13f341ce-c53d-4011-9234-d7c4626a2452 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:34.757332 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:34.839995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f341ce-c53d-4011-9234-d7c4626a2452 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:28:34.841640 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f341ce-c53d-4011-9234-d7c4626a2452 service nova] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:28:34.879339 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 691/1379] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:28:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:28:36.939935 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21678e0e-f48a-46a8-a7fa-459a08696b86 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:36.940476 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21678e0e-f48a-46a8-a7fa-459a08696b86 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:36.940476 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21678e0e-f48a-46a8-a7fa-459a08696b86 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:36.942190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 689/1380] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:28:36.947982 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea11e4-c260-4208-a870-1a0f69713195 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:36.948518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea11e4-c260-4208-a870-1a0f69713195 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:36.948612 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea11e4-c260-4208-a870-1a0f69713195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:37.020327 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-a6ea11e4-c260-4208-a870-1a0f69713195 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:37.021580 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-a6ea11e4-c260-4208-a870-1a0f69713195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:37.118468 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:37.125502 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 692/1381] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:28:37.170509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21678e0e-f48a-46a8-a7fa-459a08696b86 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:28:37.172022 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-21678e0e-f48a-46a8-a7fa-459a08696b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:37.322036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:37.334950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21678e0e-f48a-46a8-a7fa-459a08696b86 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:28:37.460178 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d73306-8a2f-4f78-a5f4-0673ace9164f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:28:37.461219 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d73306-8a2f-4f78-a5f4-0673ace9164f service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:37.461859 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d73306-8a2f-4f78-a5f4-0673ace9164f service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:37.462374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d73306-8a2f-4f78-a5f4-0673ace9164f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:37.464234 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-a7d73306-8a2f-4f78-a5f4-0673ace9164f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:37.561451 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d73306-8a2f-4f78-a5f4-0673ace9164f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:28:37.562966 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d73306-8a2f-4f78-a5f4-0673ace9164f service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:28:37.574633 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:37.584656 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 690/1382] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:28:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:28:39.667098 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaaa9b6-8505-450d-b3ea-2f5d33026bdb None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:39.667934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaaa9b6-8505-450d-b3ea-2f5d33026bdb None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:39.667934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaaa9b6-8505-450d-b3ea-2f5d33026bdb None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:39.669132 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 693/1383] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:28:39.674766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb4a6e-badf-4998-acb8-e043e2905988 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:39.675985 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb4a6e-badf-4998-acb8-e043e2905988 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:39.675985 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb4a6e-badf-4998-acb8-e043e2905988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:39.742044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-49eb4a6e-badf-4998-acb8-e043e2905988 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:39.743310 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-49eb4a6e-badf-4998-acb8-e043e2905988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:39.884297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 691/1384] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:28:39.928923 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaaa9b6-8505-450d-b3ea-2f5d33026bdb None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:28:39.931224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-deaaa9b6-8505-450d-b3ea-2f5d33026bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:40.114793 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaaa9b6-8505-450d-b3ea-2f5d33026bdb None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:28:40.188341 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:40.242598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780814b-3caa-4a01-90ff-fa14a80be696 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:28:40.243214 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780814b-3caa-4a01-90ff-fa14a80be696 service nova] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:40.243493 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780814b-3caa-4a01-90ff-fa14a80be696 service nova] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:40.243811 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780814b-3caa-4a01-90ff-fa14a80be696 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:40.245401 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-0780814b-3caa-4a01-90ff-fa14a80be696 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:40.337816 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780814b-3caa-4a01-90ff-fa14a80be696 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:28:40.339415 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780814b-3caa-4a01-90ff-fa14a80be696 service nova] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:28:40.357284 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 694/1385] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:28:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:28:42.348589 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12714272-dd37-493f-913c-7dcbaf7f7702 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:42.349177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12714272-dd37-493f-913c-7dcbaf7f7702 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:42.349302 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12714272-dd37-493f-913c-7dcbaf7f7702 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:42.350955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 692/1386] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:28:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:28:42.357037 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4deec27-1b56-4bbd-a0b0-cb0470d74848 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:42.357433 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4deec27-1b56-4bbd-a0b0-cb0470d74848 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:42.357591 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4deec27-1b56-4bbd-a0b0-cb0470d74848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:42.436664 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-f4deec27-1b56-4bbd-a0b0-cb0470d74848 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:42.438122 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f4deec27-1b56-4bbd-a0b0-cb0470d74848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:42.476753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:42.555198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 695/1387] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:28:42.610641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12714272-dd37-493f-913c-7dcbaf7f7702 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:28:42.612167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-12714272-dd37-493f-913c-7dcbaf7f7702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:42.703411 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:42.790492 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12714272-dd37-493f-913c-7dcbaf7f7702 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:28:42.932902 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e203d-5b44-4326-aee1-7a3243cc14b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:28:42.933723 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e203d-5b44-4326-aee1-7a3243cc14b7 service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:42.934093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e203d-5b44-4326-aee1-7a3243cc14b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:42.934451 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e203d-5b44-4326-aee1-7a3243cc14b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:42.936620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-211e203d-5b44-4326-aee1-7a3243cc14b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:42.986159 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:43.052489 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e203d-5b44-4326-aee1-7a3243cc14b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:28:43.054553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e203d-5b44-4326-aee1-7a3243cc14b7 service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:28:43.078257 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 693/1388] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:28:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:28:45.145834 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b5836-19a5-4a24-80f2-e391c80deae9 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:45.146369 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b5836-19a5-4a24-80f2-e391c80deae9 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:45.146369 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b5836-19a5-4a24-80f2-e391c80deae9 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:45.147751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 696/1389] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:28:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:28:45.153031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07179da0-05e4-48a3-b5df-4d9fb9cd283a None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:45.153472 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07179da0-05e4-48a3-b5df-4d9fb9cd283a None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:45.153639 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07179da0-05e4-48a3-b5df-4d9fb9cd283a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:45.221537 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-07179da0-05e4-48a3-b5df-4d9fb9cd283a None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:45.223374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-07179da0-05e4-48a3-b5df-4d9fb9cd283a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:45.321520 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 694/1390] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:28:45.366671 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b5836-19a5-4a24-80f2-e391c80deae9 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:28:45.368190 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f35b5836-19a5-4a24-80f2-e391c80deae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:45.512477 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b5836-19a5-4a24-80f2-e391c80deae9 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:28:45.530769 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:45.635807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac705f35-1fed-4719-b7eb-3ec56115da0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:28:45.636800 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac705f35-1fed-4719-b7eb-3ec56115da0d service nova] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:45.637286 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac705f35-1fed-4719-b7eb-3ec56115da0d service nova] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:45.637720 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac705f35-1fed-4719-b7eb-3ec56115da0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:45.640649 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-ac705f35-1fed-4719-b7eb-3ec56115da0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:45.731253 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac705f35-1fed-4719-b7eb-3ec56115da0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:28:45.732855 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac705f35-1fed-4719-b7eb-3ec56115da0d service nova] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:28:45.751178 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 697/1391] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:28:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:28:46.428165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab059d56-2d18-4382-9e71-90928a0892aa None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:46.428696 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab059d56-2d18-4382-9e71-90928a0892aa None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:46.428696 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab059d56-2d18-4382-9e71-90928a0892aa None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:46.430097 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 695/1392] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:28:46.435312 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c76a9-3cc8-48d6-b64b-6923a58e4292 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:46.435730 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c76a9-3cc8-48d6-b64b-6923a58e4292 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:46.435876 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c76a9-3cc8-48d6-b64b-6923a58e4292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:46.464075 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:46.506372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-ce3c76a9-3cc8-48d6-b64b-6923a58e4292 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:46.507587 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-ce3c76a9-3cc8-48d6-b64b-6923a58e4292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:46.604798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 698/1393] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:28:46.662635 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b2a3e-8ebe-4bb5-9b11-00a0389fd096 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:46.662993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b2a3e-8ebe-4bb5-9b11-00a0389fd096 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:46.663340 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b2a3e-8ebe-4bb5-9b11-00a0389fd096 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:46.664779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 696/1394] 10.222.0.74 () {54 vars in 966 bytes} [Wed Jul 27 16:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:28:46.669824 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a6873-cb77-4fc3-852c-29237235bcaf None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:46.670116 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a6873-cb77-4fc3-852c-29237235bcaf None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:46.670515 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a6873-cb77-4fc3-852c-29237235bcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:46.746885 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-052a6873-cb77-4fc3-852c-29237235bcaf None None] MFA Rules not processed for user `c50c2d1781b04a5fa8be81bce9205747`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:46.748088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-052a6873-cb77-4fc3-852c-29237235bcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:46.788173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:46.825494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 699/1395] 10.222.0.74 () {58 vars in 1063 bytes} [Wed Jul 27 16:28:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:28:46.833278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b2a3e-8ebe-4bb5-9b11-00a0389fd096 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:28:46.834921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d34b2a3e-8ebe-4bb5-9b11-00a0389fd096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:46.848624 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:46.941075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b2a3e-8ebe-4bb5-9b11-00a0389fd096 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:28:47.015088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7c7d3-b8be-4fc1-a879-6ae321a66017 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c50c2d1781b04a5fa8be81bce9205747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:28:47.015638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7c7d3-b8be-4fc1-a879-6ae321a66017 service glance] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:47.015950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7c7d3-b8be-4fc1-a879-6ae321a66017 service glance] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:47.016224 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7c7d3-b8be-4fc1-a879-6ae321a66017 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:47.017904 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-8ff7c7d3-b8be-4fc1-a879-6ae321a66017 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:47.090174 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:47.132200 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff7c7d3-b8be-4fc1-a879-6ae321a66017 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:28:47.133672 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff7c7d3-b8be-4fc1-a879-6ae321a66017 service glance] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:28:47.155121 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 697/1396] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:28:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:28:47.780096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03a1a7-fce1-4b01-b159-d406708ee855 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:47.780636 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03a1a7-fce1-4b01-b159-d406708ee855 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:47.780636 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03a1a7-fce1-4b01-b159-d406708ee855 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:47.782628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 700/1397] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:28:47.788295 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6898af-57c2-444d-8492-c1fc916874ea None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:47.788809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6898af-57c2-444d-8492-c1fc916874ea None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:47.788948 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6898af-57c2-444d-8492-c1fc916874ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:47.858793 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-8c6898af-57c2-444d-8492-c1fc916874ea None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:47.863311 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-8c6898af-57c2-444d-8492-c1fc916874ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:47.888601 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cec3b-032d-4f16-b3c5-1f7b102ee0e2 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:47.888832 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cec3b-032d-4f16-b3c5-1f7b102ee0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:47.889147 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cec3b-032d-4f16-b3c5-1f7b102ee0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:47.957351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-644cec3b-032d-4f16-b3c5-1f7b102ee0e2 None None] MFA Rules not processed for user `07d6bffc69674103b4c6540598a780b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:47.958557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-644cec3b-032d-4f16-b3c5-1f7b102ee0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:47.963929 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 698/1398] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:28:48.008131 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6898af-57c2-444d-8492-c1fc916874ea None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:28:48.009498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-8c6898af-57c2-444d-8492-c1fc916874ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:48.036297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:48.048443 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 701/1399] 10.222.0.74 () {58 vars in 1009 bytes} [Wed Jul 27 16:28:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:28:48.096418 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d633ae-2bb2-466e-abd8-192f030987ec None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:48.096706 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d633ae-2bb2-466e-abd8-192f030987ec None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:48.097053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d633ae-2bb2-466e-abd8-192f030987ec None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:48.098408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 702/1400] 10.222.0.74 () {54 vars in 943 bytes} [Wed Jul 27 16:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:28:48.104364 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1c120-8804-469f-b6df-7d5e2af8a466 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:48.104718 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1c120-8804-469f-b6df-7d5e2af8a466 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:48.105045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1c120-8804-469f-b6df-7d5e2af8a466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:48.131454 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6898af-57c2-444d-8492-c1fc916874ea None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:28:48.174939 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:48.182922 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-9ac1c120-8804-469f-b6df-7d5e2af8a466 None None] MFA Rules not processed for user `bb45ded9f75e4fa2973b564f83bdfaf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:48.184130 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-9ac1c120-8804-469f-b6df-7d5e2af8a466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:48.251970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a7702-b80e-47cc-89fd-c70f4c7a7d4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:28:48.252594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a7702-b80e-47cc-89fd-c70f4c7a7d4d service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:48.252880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a7702-b80e-47cc-89fd-c70f4c7a7d4d service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:48.253198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a7702-b80e-47cc-89fd-c70f4c7a7d4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:48.254793 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-541a7702-b80e-47cc-89fd-c70f4c7a7d4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:48.273536 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:48.297096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 703/1401] 10.222.0.74 () {58 vars in 1040 bytes} [Wed Jul 27 16:28:48 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:28:48.301514 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1c120-8804-469f-b6df-7d5e2af8a466 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:28:48.310735 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1c120-8804-469f-b6df-7d5e2af8a466 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:28:48.347580 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a7702-b80e-47cc-89fd-c70f4c7a7d4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:28:48.349229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a7702-b80e-47cc-89fd-c70f4c7a7d4d service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:28:48.367078 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 699/1402] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:28:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:28:48.432164 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56945073-b637-42eb-a7f5-7dee57b445ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb45ded9f75e4fa2973b564f83bdfaf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:28:48.432838 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56945073-b637-42eb-a7f5-7dee57b445ad service swift] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:48.433152 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56945073-b637-42eb-a7f5-7dee57b445ad service swift] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:48.433422 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56945073-b637-42eb-a7f5-7dee57b445ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:48.436001 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-56945073-b637-42eb-a7f5-7dee57b445ad service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:48.438664 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:48.533010 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56945073-b637-42eb-a7f5-7dee57b445ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:28:48.534850 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56945073-b637-42eb-a7f5-7dee57b445ad service swift] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:28:48.548818 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 704/1403] 10.222.0.74 () {60 vars in 1384 bytes} [Wed Jul 27 16:28:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 27 16:28:50.458810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f900d6-a98e-4ed0-bad5-cc90efff684d None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:50.459445 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f900d6-a98e-4ed0-bad5-cc90efff684d None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:50.459445 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f900d6-a98e-4ed0-bad5-cc90efff684d None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:50.460699 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 700/1404] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:28:50.465924 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950d38d-9c08-46af-974f-524692b7b228 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:50.466355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950d38d-9c08-46af-974f-524692b7b228 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:50.466490 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950d38d-9c08-46af-974f-524692b7b228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:50.537485 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-e950d38d-9c08-46af-974f-524692b7b228 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:50.539018 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e950d38d-9c08-46af-974f-524692b7b228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:50.649865 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:50.657522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 705/1405] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:28:50.702518 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f900d6-a98e-4ed0-bad5-cc90efff684d None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:28:50.704184 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-51f900d6-a98e-4ed0-bad5-cc90efff684d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:50.877785 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:50.880659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f900d6-a98e-4ed0-bad5-cc90efff684d None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:28:51.012938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53c71f-0642-4cef-8e3c-c557d1f089ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:28:51.013658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53c71f-0642-4cef-8e3c-c557d1f089ac service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:51.014025 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53c71f-0642-4cef-8e3c-c557d1f089ac service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:51.014390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53c71f-0642-4cef-8e3c-c557d1f089ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:51.016203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6f53c71f-0642-4cef-8e3c-c557d1f089ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:51.114702 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f53c71f-0642-4cef-8e3c-c557d1f089ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:28:51.116310 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f53c71f-0642-4cef-8e3c-c557d1f089ac service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:28:51.135984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 701/1406] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:28:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:28:53.210873 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e523d25-cc7e-481d-8bdf-9feb0710de29 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:53.211422 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e523d25-cc7e-481d-8bdf-9feb0710de29 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:53.211422 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e523d25-cc7e-481d-8bdf-9feb0710de29 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:53.212788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 706/1407] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:28:53.217947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c843456-d333-40cc-a5c3-5cce9017103c None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:53.218354 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c843456-d333-40cc-a5c3-5cce9017103c None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:53.218535 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c843456-d333-40cc-a5c3-5cce9017103c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:53.228808 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:53.291987 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-6c843456-d333-40cc-a5c3-5cce9017103c None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:28:53.293287 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6c843456-d333-40cc-a5c3-5cce9017103c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:53.397846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 702/1408] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:28:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:28:53.442509 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e523d25-cc7e-481d-8bdf-9feb0710de29 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:28:53.444037 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-4e523d25-cc7e-481d-8bdf-9feb0710de29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:53.607771 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e523d25-cc7e-481d-8bdf-9feb0710de29 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:28:53.679046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:28:53.736771 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5832815-6512-4817-872a-a9a20b543429 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:28:53.737472 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5832815-6512-4817-872a-a9a20b543429 service nova] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:28:53.737755 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5832815-6512-4817-872a-a9a20b543429 service nova] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:28:53.738028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5832815-6512-4817-872a-a9a20b543429 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:28:53.739694 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d5832815-6512-4817-872a-a9a20b543429 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:28:53.854065 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5832815-6512-4817-872a-a9a20b543429 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:28:53.855821 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5832815-6512-4817-872a-a9a20b543429 service nova] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:28:53.879494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 707/1409] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:28:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:29:03.395696 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=64 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:03.875764 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=64 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:10.817917 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7ff55-653c-4078-96b5-198858d5ab24 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:10.818519 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7ff55-653c-4078-96b5-198858d5ab24 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:10.818519 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7ff55-653c-4078-96b5-198858d5ab24 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:10.819878 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 703/1410] 10.222.0.74 () {54 vars in 918 bytes} [Wed Jul 27 16:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:29:10.824850 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f4abd-d0ee-46fc-b16d-691339858c3e None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:10.825321 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f4abd-d0ee-46fc-b16d-691339858c3e None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:10.825430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f4abd-d0ee-46fc-b16d-691339858c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:10.900442 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-723f4abd-d0ee-46fc-b16d-691339858c3e None None] MFA Rules not processed for user `91f8e0f82a8541aead6cd8cc45f4a86b`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:29:10.901681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-723f4abd-d0ee-46fc-b16d-691339858c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:11.008382 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 708/1411] 10.222.0.74 () {58 vars in 1015 bytes} [Wed Jul 27 16:29:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:29:11.023456 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552fa39-9146-49f3-a57c-2cac91774596 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:11.023743 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552fa39-9146-49f3-a57c-2cac91774596 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:11.024024 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552fa39-9146-49f3-a57c-2cac91774596 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:11.025395 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 704/1412] 10.222.0.74 () {54 vars in 918 bytes} [Wed Jul 27 16:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:29:11.029961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4adcd-9410-4a75-9299-4f5743a720a0 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:11.030245 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4adcd-9410-4a75-9299-4f5743a720a0 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:11.030516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4adcd-9410-4a75-9299-4f5743a720a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:11.102651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:11.105651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-9cb4adcd-9410-4a75-9299-4f5743a720a0 None None] MFA Rules not processed for user `56dfafd15e284b7986ba07dd3819bf65`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:29:11.107440 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-9cb4adcd-9410-4a75-9299-4f5743a720a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:11.222502 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 709/1413] 10.222.0.74 () {58 vars in 1015 bytes} [Wed Jul 27 16:29:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:29:13.748067 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93594314-af33-4c40-984f-cca3874dc711 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:13.748620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93594314-af33-4c40-984f-cca3874dc711 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:13.748620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93594314-af33-4c40-984f-cca3874dc711 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:13.749985 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 16:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:29:13.755231 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334983a-ebbb-45d0-9da7-6266e3831dad None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:13.755672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334983a-ebbb-45d0-9da7-6266e3831dad None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:13.755805 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334983a-ebbb-45d0-9da7-6266e3831dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:13.821852 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-3334983a-ebbb-45d0-9da7-6266e3831dad None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:29:13.823062 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-3334983a-ebbb-45d0-9da7-6266e3831dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:13.878291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:13.923042 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 16:29:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:29:13.966551 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93594314-af33-4c40-984f-cca3874dc711 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:29:13.968038 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-93594314-af33-4c40-984f-cca3874dc711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:14.111376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93594314-af33-4c40-984f-cca3874dc711 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:29:14.184208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:14.226493 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b263a5-d803-479c-a820-928c4826bd62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:29:14.227110 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b263a5-d803-479c-a820-928c4826bd62 service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:14.227388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b263a5-d803-479c-a820-928c4826bd62 service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:14.227655 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b263a5-d803-479c-a820-928c4826bd62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:14.229277 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-87b263a5-d803-479c-a820-928c4826bd62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:14.314844 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b263a5-d803-479c-a820-928c4826bd62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:29:14.316449 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b263a5-d803-479c-a820-928c4826bd62 service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:29:14.335093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 706/1416] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:29:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:29:15.266620 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129f399-6361-487f-a1ca-7abf6c563ea9 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:15.267195 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129f399-6361-487f-a1ca-7abf6c563ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:15.267195 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129f399-6361-487f-a1ca-7abf6c563ea9 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:15.269095 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 16:29:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:29:15.274003 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f0812-47df-4570-96ee-8d44431f5394 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:15.274437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f0812-47df-4570-96ee-8d44431f5394 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:15.274548 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f0812-47df-4570-96ee-8d44431f5394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:15.349601 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-353f0812-47df-4570-96ee-8d44431f5394 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:29:15.350808 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-353f0812-47df-4570-96ee-8d44431f5394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:15.376964 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:15.448995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 16:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:29:15.601111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129f399-6361-487f-a1ca-7abf6c563ea9 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:29:15.602672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e129f399-6361-487f-a1ca-7abf6c563ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:15.749003 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129f399-6361-487f-a1ca-7abf6c563ea9 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:29:15.796116 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:15.870843 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38161d7-3a97-479a-8c22-079b7c0772f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:29:15.871456 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38161d7-3a97-479a-8c22-079b7c0772f3 service nova] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:15.871739 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38161d7-3a97-479a-8c22-079b7c0772f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:15.872006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38161d7-3a97-479a-8c22-079b7c0772f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:15.873642 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d38161d7-3a97-479a-8c22-079b7c0772f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:15.962479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38161d7-3a97-479a-8c22-079b7c0772f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:29:15.964074 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38161d7-3a97-479a-8c22-079b7c0772f3 service nova] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:29:15.981600 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 712/1419] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:29:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:29:16.865718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eee8e1-d354-483e-9824-c28681108815 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:16.866279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eee8e1-d354-483e-9824-c28681108815 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:16.866279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eee8e1-d354-483e-9824-c28681108815 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:16.867609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 16:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:29:16.872409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e79737-3712-48d0-97df-424cc4553c60 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:16.872809 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e79737-3712-48d0-97df-424cc4553c60 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:16.873003 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e79737-3712-48d0-97df-424cc4553c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:16.928007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:16.951196 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-70e79737-3712-48d0-97df-424cc4553c60 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:29:16.952409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-70e79737-3712-48d0-97df-424cc4553c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:17.047938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 16:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:29:17.200924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eee8e1-d354-483e-9824-c28681108815 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:29:17.202369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d8eee8e1-d354-483e-9824-c28681108815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:17.349107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eee8e1-d354-483e-9824-c28681108815 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:29:17.360858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:17.459720 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229670f-9560-447a-8a4f-1bfd9ac75de5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:29:17.460333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229670f-9560-447a-8a4f-1bfd9ac75de5 service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:17.460611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229670f-9560-447a-8a4f-1bfd9ac75de5 service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:17.460904 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229670f-9560-447a-8a4f-1bfd9ac75de5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:17.462522 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-2229670f-9560-447a-8a4f-1bfd9ac75de5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:17.549100 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229670f-9560-447a-8a4f-1bfd9ac75de5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:29:17.550718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229670f-9560-447a-8a4f-1bfd9ac75de5 service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:29:17.568100 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 709/1422] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:29:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:29:18.430385 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5ac88-6433-4d72-a249-29e2fc5965b7 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:18.430873 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5ac88-6433-4d72-a249-29e2fc5965b7 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:18.430873 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5ac88-6433-4d72-a249-29e2fc5965b7 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:18.432236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 16:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:29:18.437209 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c215f0c-5025-468e-a9ed-a4e6754e238f None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:18.437621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c215f0c-5025-468e-a9ed-a4e6754e238f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:18.437775 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c215f0c-5025-468e-a9ed-a4e6754e238f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:18.458499 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:18.513951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-7c215f0c-5025-468e-a9ed-a4e6754e238f None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:29:18.515156 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-7c215f0c-5025-468e-a9ed-a4e6754e238f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:18.615178 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 16:29:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:29:18.765597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5ac88-6433-4d72-a249-29e2fc5965b7 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:29:18.767136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7ac5ac88-6433-4d72-a249-29e2fc5965b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:18.877877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:18.913978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5ac88-6433-4d72-a249-29e2fc5965b7 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:29:19.024755 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1687e-b4b5-4619-8a48-67f444a88f56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:29:19.025338 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1687e-b4b5-4619-8a48-67f444a88f56 service nova] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:19.025649 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1687e-b4b5-4619-8a48-67f444a88f56 service nova] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:19.025924 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1687e-b4b5-4619-8a48-67f444a88f56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:19.027680 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-9af1687e-b4b5-4619-8a48-67f444a88f56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:19.103225 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:19.117250 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1687e-b4b5-4619-8a48-67f444a88f56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:29:19.118854 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af1687e-b4b5-4619-8a48-67f444a88f56 service nova] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:29:19.137266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 715/1425] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:29:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:29:20.001470 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4f5ad-961b-4901-83ba-c2d99af6c334 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:20.002121 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4f5ad-961b-4901-83ba-c2d99af6c334 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:20.002121 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4f5ad-961b-4901-83ba-c2d99af6c334 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:20.003376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 16:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:29:20.008415 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef08a2-84dc-4165-b4a3-d78218d17bfb None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:20.008817 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef08a2-84dc-4165-b4a3-d78218d17bfb None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:20.009003 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef08a2-84dc-4165-b4a3-d78218d17bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:20.084436 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-25ef08a2-84dc-4165-b4a3-d78218d17bfb None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:29:20.085676 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-25ef08a2-84dc-4165-b4a3-d78218d17bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:20.185423 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 16:29:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:29:20.334223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4f5ad-961b-4901-83ba-c2d99af6c334 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:29:20.335721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-8da4f5ad-961b-4901-83ba-c2d99af6c334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:20.417234 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:20.479864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4f5ad-961b-4901-83ba-c2d99af6c334 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:29:20.589357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1130e1-e04f-4c00-8825-b6b0c265e917 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:29:20.589888 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1130e1-e04f-4c00-8825-b6b0c265e917 service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:20.590164 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1130e1-e04f-4c00-8825-b6b0c265e917 service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:20.590432 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1130e1-e04f-4c00-8825-b6b0c265e917 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:20.592028 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ea1130e1-e04f-4c00-8825-b6b0c265e917 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:20.640534 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:20.681099 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1130e1-e04f-4c00-8825-b6b0c265e917 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:29:20.682693 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1130e1-e04f-4c00-8825-b6b0c265e917 service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:29:20.700682 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 712/1428] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:29:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:29:21.491208 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64625e-7053-4986-8a73-f68b3de5e09f None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:21.491759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64625e-7053-4986-8a73-f68b3de5e09f None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:21.491759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64625e-7053-4986-8a73-f68b3de5e09f None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:21.493250 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 16:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:29:21.498268 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b38b61-0489-4dc5-8716-a1b629a84882 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:21.498683 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b38b61-0489-4dc5-8716-a1b629a84882 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:21.498826 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b38b61-0489-4dc5-8716-a1b629a84882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:21.573422 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-41b38b61-0489-4dc5-8716-a1b629a84882 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:29:21.574615 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-41b38b61-0489-4dc5-8716-a1b629a84882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:21.670586 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 16:29:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:29:21.821224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64625e-7053-4986-8a73-f68b3de5e09f None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:29:21.822692 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-bd64625e-7053-4986-8a73-f68b3de5e09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:21.872530 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:22.096197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64625e-7053-4986-8a73-f68b3de5e09f None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:29:22.230336 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75a4e7-0fee-42c7-9a42-43f9e6ee29d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:29:22.230954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75a4e7-0fee-42c7-9a42-43f9e6ee29d6 service nova] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:22.231236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75a4e7-0fee-42c7-9a42-43f9e6ee29d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:22.231510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75a4e7-0fee-42c7-9a42-43f9e6ee29d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:22.233872 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-fd75a4e7-0fee-42c7-9a42-43f9e6ee29d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:22.243585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:22.325263 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75a4e7-0fee-42c7-9a42-43f9e6ee29d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:29:22.326832 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75a4e7-0fee-42c7-9a42-43f9e6ee29d6 service nova] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:29:22.344875 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 718/1431] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:29:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:29:23.141541 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2031384-3b9f-402b-832c-819f0b1e3d79 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:23.142112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2031384-3b9f-402b-832c-819f0b1e3d79 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:23.142112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2031384-3b9f-402b-832c-819f0b1e3d79 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:23.144417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 16:29:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:29:23.148916 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e59fe-85c7-4b00-81c2-90ebbade6329 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:23.149345 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e59fe-85c7-4b00-81c2-90ebbade6329 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:23.149478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e59fe-85c7-4b00-81c2-90ebbade6329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:23.225020 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-286e59fe-85c7-4b00-81c2-90ebbade6329 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:29:23.226220 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-286e59fe-85c7-4b00-81c2-90ebbade6329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:23.292433 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:23.321600 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 16:29:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:29:23.470731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2031384-3b9f-402b-832c-819f0b1e3d79 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:29:23.472820 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b2031384-3b9f-402b-832c-819f0b1e3d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:23.490079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:23.616608 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2031384-3b9f-402b-832c-819f0b1e3d79 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:29:23.703807 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:23.730279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19fceb2-e24f-43f0-81a7-306b16e99abb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:29:23.730837 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fceb2-e24f-43f0-81a7-306b16e99abb service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:23.731119 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fceb2-e24f-43f0-81a7-306b16e99abb service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:23.731395 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fceb2-e24f-43f0-81a7-306b16e99abb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:23.733038 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-f19fceb2-e24f-43f0-81a7-306b16e99abb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:23.820309 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19fceb2-e24f-43f0-81a7-306b16e99abb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:29:23.821935 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19fceb2-e24f-43f0-81a7-306b16e99abb service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:29:23.840283 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 715/1434] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:29:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:29:24.647120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7d3e4-096b-4727-9abf-142a0e52ede2 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:24.647656 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7d3e4-096b-4727-9abf-142a0e52ede2 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:24.647656 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7d3e4-096b-4727-9abf-142a0e52ede2 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:24.648997 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 16:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:29:24.653816 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98a7e6-4dd3-4883-b78f-72fa7bae15d4 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:24.654214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98a7e6-4dd3-4883-b78f-72fa7bae15d4 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:24.654394 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98a7e6-4dd3-4883-b78f-72fa7bae15d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:24.728622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-ce98a7e6-4dd3-4883-b78f-72fa7bae15d4 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:29:24.729856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ce98a7e6-4dd3-4883-b78f-72fa7bae15d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:24.771524 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:24.825850 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 16:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:29:24.976558 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7d3e4-096b-4727-9abf-142a0e52ede2 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:29:24.978089 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-70b7d3e4-096b-4727-9abf-142a0e52ede2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:25.128135 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7d3e4-096b-4727-9abf-142a0e52ede2 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:29:25.181866 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:25.239822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf1f7ba-b372-4000-98db-769a725cd987 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:29:25.240407 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1f7ba-b372-4000-98db-769a725cd987 service nova] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:25.240684 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1f7ba-b372-4000-98db-769a725cd987 service nova] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:25.240984 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1f7ba-b372-4000-98db-769a725cd987 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:25.242592 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-2bf1f7ba-b372-4000-98db-769a725cd987 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:25.335257 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf1f7ba-b372-4000-98db-769a725cd987 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:29:25.336852 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf1f7ba-b372-4000-98db-769a725cd987 service nova] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:29:25.354600 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 721/1437] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:29:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:29:26.171054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfc79e-9250-4151-8712-a87bada6dc0f None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:26.171481 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfc79e-9250-4151-8712-a87bada6dc0f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:26.171619 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfc79e-9250-4151-8712-a87bada6dc0f None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:26.173025 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 16:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:29:26.178523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cf483-b6f4-4103-b5af-60e5fd748d1b None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:26.178950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cf483-b6f4-4103-b5af-60e5fd748d1b None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:26.179081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cf483-b6f4-4103-b5af-60e5fd748d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:26.253514 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-9a7cf483-b6f4-4103-b5af-60e5fd748d1b None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:29:26.254704 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-9a7cf483-b6f4-4103-b5af-60e5fd748d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:26.256682 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:26.352226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 16:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:29:26.502320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbfc79e-9250-4151-8712-a87bada6dc0f None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:29:26.504374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0dbfc79e-9250-4151-8712-a87bada6dc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:26.646538 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbfc79e-9250-4151-8712-a87bada6dc0f None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:29:26.672709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:26.758817 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7da81-a748-486b-b509-5f544d3758f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:29:26.759357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7da81-a748-486b-b509-5f544d3758f6 service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:26.759633 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7da81-a748-486b-b509-5f544d3758f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:26.759904 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7da81-a748-486b-b509-5f544d3758f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:26.761517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-10d7da81-a748-486b-b509-5f544d3758f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:26.850996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7da81-a748-486b-b509-5f544d3758f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:29:26.852643 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7da81-a748-486b-b509-5f544d3758f6 service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:29:26.872116 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 718/1440] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:29:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:29:28.807779 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2444d1-3e97-4b45-8a1c-6cd337bdcc4c None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:28.808295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2444d1-3e97-4b45-8a1c-6cd337bdcc4c None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:28.808295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2444d1-3e97-4b45-8a1c-6cd337bdcc4c None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:28.809676 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 16:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:29:28.815653 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd88eb-518a-4542-a82d-e8f9559ca98f None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:28.816093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd88eb-518a-4542-a82d-e8f9559ca98f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:28.816203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd88eb-518a-4542-a82d-e8f9559ca98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:28.856339 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:28.882835 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-97cd88eb-518a-4542-a82d-e8f9559ca98f None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:29:28.884043 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-97cd88eb-518a-4542-a82d-e8f9559ca98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:28.983562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 16:29:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:29:29.022145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2444d1-3e97-4b45-8a1c-6cd337bdcc4c None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:29:29.023682 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-8e2444d1-3e97-4b45-8a1c-6cd337bdcc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:29.125996 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2444d1-3e97-4b45-8a1c-6cd337bdcc4c None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:29:29.156929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:29.198640 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec179c31-ea8a-46c1-9dba-f3b0975968c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:29:29.199256 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec179c31-ea8a-46c1-9dba-f3b0975968c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:29.199534 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec179c31-ea8a-46c1-9dba-f3b0975968c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:29.199802 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec179c31-ea8a-46c1-9dba-f3b0975968c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:29.201431 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-ec179c31-ea8a-46c1-9dba-f3b0975968c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:29.308227 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec179c31-ea8a-46c1-9dba-f3b0975968c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:29:29.309672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec179c31-ea8a-46c1-9dba-f3b0975968c1 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:29:29.327062 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 724/1443] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:29:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:29:31.458628 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0b129-b360-4e81-a406-b38d502a1bf9 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:31.459148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0b129-b360-4e81-a406-b38d502a1bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:31.459246 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0b129-b360-4e81-a406-b38d502a1bf9 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:31.460564 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 16:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:29:31.465688 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25180656-4c75-4cda-b1cd-06f0398289e3 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:31.466113 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25180656-4c75-4cda-b1cd-06f0398289e3 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:31.466241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25180656-4c75-4cda-b1cd-06f0398289e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:31.531421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-25180656-4c75-4cda-b1cd-06f0398289e3 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:29:31.532614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-25180656-4c75-4cda-b1cd-06f0398289e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:31.545813 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:31.631280 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 16:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:29:31.669423 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0b129-b360-4e81-a406-b38d502a1bf9 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:29:31.670998 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b4c0b129-b360-4e81-a406-b38d502a1bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:31.772971 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0b129-b360-4e81-a406-b38d502a1bf9 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:29:31.785369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:31.844916 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9dcd34-a5c3-450c-b174-3586fed6be32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:29:31.845562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9dcd34-a5c3-450c-b174-3586fed6be32 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:31.845826 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9dcd34-a5c3-450c-b174-3586fed6be32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:31.846681 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9dcd34-a5c3-450c-b174-3586fed6be32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:31.848293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-5c9dcd34-a5c3-450c-b174-3586fed6be32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:31.955166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9dcd34-a5c3-450c-b174-3586fed6be32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:29:31.957258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9dcd34-a5c3-450c-b174-3586fed6be32 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:29:31.974774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 721/1446] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:29:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:29:34.290933 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da19d5-138b-4d1c-bab4-0faf1f7cc73e None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:34.291474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da19d5-138b-4d1c-bab4-0faf1f7cc73e None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:34.291474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da19d5-138b-4d1c-bab4-0faf1f7cc73e None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:34.292838 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 16:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:29:34.298211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea227df-3555-472d-a887-27b1d12572d5 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:34.298644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea227df-3555-472d-a887-27b1d12572d5 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:34.298757 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea227df-3555-472d-a887-27b1d12572d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:34.347144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:34.365015 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-cea227df-3555-472d-a887-27b1d12572d5 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:29:34.366239 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-cea227df-3555-472d-a887-27b1d12572d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:34.461716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 16:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:29:34.500552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da19d5-138b-4d1c-bab4-0faf1f7cc73e None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:29:34.502166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-38da19d5-138b-4d1c-bab4-0faf1f7cc73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:34.605267 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da19d5-138b-4d1c-bab4-0faf1f7cc73e None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:29:34.649793 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:34.683520 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26fb10-1d59-413b-b015-6428a57dfaa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:29:34.684131 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26fb10-1d59-413b-b015-6428a57dfaa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:34.684404 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26fb10-1d59-413b-b015-6428a57dfaa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:34.684669 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26fb10-1d59-413b-b015-6428a57dfaa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:34.686290 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-ea26fb10-1d59-413b-b015-6428a57dfaa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:34.793300 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea26fb10-1d59-413b-b015-6428a57dfaa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:29:34.794648 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea26fb10-1d59-413b-b015-6428a57dfaa0 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:29:34.813441 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 727/1449] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:29:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:29:36.957719 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04dd66-7333-492a-a200-0f082398f9f6 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:36.958310 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04dd66-7333-492a-a200-0f082398f9f6 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:36.958310 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04dd66-7333-492a-a200-0f082398f9f6 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:36.959621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 16:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:29:36.964750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a0ed0-7a52-46a9-ae5c-144ba400e4f1 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:36.965168 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a0ed0-7a52-46a9-ae5c-144ba400e4f1 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:36.965331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a0ed0-7a52-46a9-ae5c-144ba400e4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:37.030407 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-eb7a0ed0-7a52-46a9-ae5c-144ba400e4f1 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:29:37.031606 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-eb7a0ed0-7a52-46a9-ae5c-144ba400e4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:37.052035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:37.129113 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 16:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:29:37.167038 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd04dd66-7333-492a-a200-0f082398f9f6 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:29:37.168614 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-fd04dd66-7333-492a-a200-0f082398f9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:37.272325 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd04dd66-7333-492a-a200-0f082398f9f6 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:29:37.291301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:37.344426 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adedffb-0c89-4710-85cc-a4271293d8fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:29:37.345085 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adedffb-0c89-4710-85cc-a4271293d8fe service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:37.345368 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adedffb-0c89-4710-85cc-a4271293d8fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:37.345641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adedffb-0c89-4710-85cc-a4271293d8fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:37.347496 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-4adedffb-0c89-4710-85cc-a4271293d8fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:37.453764 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adedffb-0c89-4710-85cc-a4271293d8fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:29:37.455195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adedffb-0c89-4710-85cc-a4271293d8fe service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:29:37.473397 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 724/1452] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:29:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:29:39.515241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cb962-3dac-41d3-b1c5-3b4c5b96212d None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:39.515807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cb962-3dac-41d3-b1c5-3b4c5b96212d None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:39.515807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cb962-3dac-41d3-b1c5-3b4c5b96212d None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:39.517183 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 16:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:29:39.523141 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aaa5f7-bcf7-488c-b62e-9886392ff81b None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:39.523543 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aaa5f7-bcf7-488c-b62e-9886392ff81b None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:39.523787 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aaa5f7-bcf7-488c-b62e-9886392ff81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:39.586804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:39.589788 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-e5aaa5f7-bcf7-488c-b62e-9886392ff81b None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:29:39.591025 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-e5aaa5f7-bcf7-488c-b62e-9886392ff81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:39.686568 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 16:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:29:39.725319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cb962-3dac-41d3-b1c5-3b4c5b96212d None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:29:39.726858 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-ce6cb962-3dac-41d3-b1c5-3b4c5b96212d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:39.828726 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cb962-3dac-41d3-b1c5-3b4c5b96212d None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:29:39.876867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:39.902718 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a69f13-a68f-4d95-a306-01fbdfc85606 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:29:39.903331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a69f13-a68f-4d95-a306-01fbdfc85606 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:39.903613 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a69f13-a68f-4d95-a306-01fbdfc85606 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:39.903892 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a69f13-a68f-4d95-a306-01fbdfc85606 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:39.905537 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-64a69f13-a68f-4d95-a306-01fbdfc85606 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:40.013913 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a69f13-a68f-4d95-a306-01fbdfc85606 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:29:40.015350 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a69f13-a68f-4d95-a306-01fbdfc85606 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:29:40.033337 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 730/1455] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:29:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:29:42.067435 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b79cdc-aef9-4514-98d4-12340dd2bdb7 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:42.067962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b79cdc-aef9-4514-98d4-12340dd2bdb7 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:42.067962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b79cdc-aef9-4514-98d4-12340dd2bdb7 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:42.069407 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 16:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:29:42.074770 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fb427-a7d5-423f-a9f9-5311b2b65710 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:42.075163 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fb427-a7d5-423f-a9f9-5311b2b65710 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:42.075327 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fb427-a7d5-423f-a9f9-5311b2b65710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:42.141163 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-100fb427-a7d5-423f-a9f9-5311b2b65710 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:29:42.142368 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-100fb427-a7d5-423f-a9f9-5311b2b65710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:42.176110 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:42.242448 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 16:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:29:42.280281 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b79cdc-aef9-4514-98d4-12340dd2bdb7 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:29:42.281889 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-32b79cdc-aef9-4514-98d4-12340dd2bdb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:42.385831 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b79cdc-aef9-4514-98d4-12340dd2bdb7 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:29:42.415747 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:42.461975 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e73e55-e896-4123-8327-2e38fec16888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:29:42.462591 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e73e55-e896-4123-8327-2e38fec16888 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:42.462888 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e73e55-e896-4123-8327-2e38fec16888 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:42.463154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e73e55-e896-4123-8327-2e38fec16888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:42.464773 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c9e73e55-e896-4123-8327-2e38fec16888 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:42.573125 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e73e55-e896-4123-8327-2e38fec16888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:29:42.574555 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e73e55-e896-4123-8327-2e38fec16888 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:29:42.593031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 727/1458] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:29:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:29:44.681471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b6f0b-e8c0-434e-b4f2-245ade00144a None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:44.682049 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b6f0b-e8c0-434e-b4f2-245ade00144a None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:44.682049 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b6f0b-e8c0-434e-b4f2-245ade00144a None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:44.683366 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 16:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:29:44.688448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f87df-6ff8-412d-af36-62120bdea541 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:44.688868 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f87df-6ff8-412d-af36-62120bdea541 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:44.689046 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f87df-6ff8-412d-af36-62120bdea541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:44.753745 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-fd6f87df-6ff8-412d-af36-62120bdea541 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:29:44.754948 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-fd6f87df-6ff8-412d-af36-62120bdea541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:44.763058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:44.851644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 16:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:29:44.888953 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b6f0b-e8c0-434e-b4f2-245ade00144a None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:29:44.890549 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-749b6f0b-e8c0-434e-b4f2-245ade00144a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:44.995961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b6f0b-e8c0-434e-b4f2-245ade00144a None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:29:45.051821 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:45.070831 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4bd21-57e2-4bfa-9259-dc3068de88ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:29:45.071438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4bd21-57e2-4bfa-9259-dc3068de88ca service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:45.071714 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4bd21-57e2-4bfa-9259-dc3068de88ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:45.071982 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4bd21-57e2-4bfa-9259-dc3068de88ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:45.073603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b4e4bd21-57e2-4bfa-9259-dc3068de88ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:45.181524 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e4bd21-57e2-4bfa-9259-dc3068de88ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:29:45.182987 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e4bd21-57e2-4bfa-9259-dc3068de88ca service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:29:45.201176 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 733/1461] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:29:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:29:47.443667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4a83b-92cf-4661-b31a-dc231529b788 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:47.444207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4a83b-92cf-4661-b31a-dc231529b788 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:47.444207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4a83b-92cf-4661-b31a-dc231529b788 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:47.445558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 16:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:29:47.450999 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb38cf-3c28-43b6-8ebe-a4fbf70fa341 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:47.451410 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb38cf-3c28-43b6-8ebe-a4fbf70fa341 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:47.451550 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb38cf-3c28-43b6-8ebe-a4fbf70fa341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:47.517049 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-61bb38cf-3c28-43b6-8ebe-a4fbf70fa341 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:29:47.518282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-61bb38cf-3c28-43b6-8ebe-a4fbf70fa341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:47.555466 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:47.612640 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 16:29:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:29:47.650713 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c4a83b-92cf-4661-b31a-dc231529b788 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:29:47.652260 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-75c4a83b-92cf-4661-b31a-dc231529b788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:47.754681 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c4a83b-92cf-4661-b31a-dc231529b788 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:29:47.791670 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:47.827577 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66243542-18b0-48dc-8cc5-9f0d4f8ed2f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:29:47.828205 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66243542-18b0-48dc-8cc5-9f0d4f8ed2f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:47.828496 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66243542-18b0-48dc-8cc5-9f0d4f8ed2f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:47.828774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66243542-18b0-48dc-8cc5-9f0d4f8ed2f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:47.830451 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-66243542-18b0-48dc-8cc5-9f0d4f8ed2f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:47.938713 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66243542-18b0-48dc-8cc5-9f0d4f8ed2f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:29:47.940170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66243542-18b0-48dc-8cc5-9f0d4f8ed2f5 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:29:47.957954 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 730/1464] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:29:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:29:49.968287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d166e-e1b7-44e6-9060-e2560c3bd130 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:49.968829 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d166e-e1b7-44e6-9060-e2560c3bd130 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:49.968829 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d166e-e1b7-44e6-9060-e2560c3bd130 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:49.970213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 16:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:29:49.975928 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6a0fe-744d-44db-8aef-3f88e45086d9 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:49.976328 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6a0fe-744d-44db-8aef-3f88e45086d9 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:49.976528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6a0fe-744d-44db-8aef-3f88e45086d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:50.041760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-b2a6a0fe-744d-44db-8aef-3f88e45086d9 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:29:50.042972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b2a6a0fe-744d-44db-8aef-3f88e45086d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:50.060504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:50.137492 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 16:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:29:50.174392 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d166e-e1b7-44e6-9060-e2560c3bd130 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:29:50.175972 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b62d166e-e1b7-44e6-9060-e2560c3bd130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:50.284077 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d166e-e1b7-44e6-9060-e2560c3bd130 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:29:50.350365 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:50.357928 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9fb65-42c5-4dff-afd2-bbfec160a959 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:29:50.358544 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9fb65-42c5-4dff-afd2-bbfec160a959 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:50.358821 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9fb65-42c5-4dff-afd2-bbfec160a959 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:50.359093 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9fb65-42c5-4dff-afd2-bbfec160a959 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:50.360688 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-0ff9fb65-42c5-4dff-afd2-bbfec160a959 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:50.468877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9fb65-42c5-4dff-afd2-bbfec160a959 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:29:50.470364 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9fb65-42c5-4dff-afd2-bbfec160a959 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:29:50.489538 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 736/1467] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:29:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:29:52.495555 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9548ba2-3770-4179-a133-486b220f7ed9 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:52.496164 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9548ba2-3770-4179-a133-486b220f7ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:52.496164 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9548ba2-3770-4179-a133-486b220f7ed9 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:52.497568 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 16:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:29:52.503873 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594bb56-799b-4e27-b645-cde830b828e4 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:52.504398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594bb56-799b-4e27-b645-cde830b828e4 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:52.504486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594bb56-799b-4e27-b645-cde830b828e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:52.570218 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-d594bb56-799b-4e27-b645-cde830b828e4 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:29:52.571416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d594bb56-799b-4e27-b645-cde830b828e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:52.619406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:52.667150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 16:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:29:52.704368 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9548ba2-3770-4179-a133-486b220f7ed9 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:29:52.706238 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b9548ba2-3770-4179-a133-486b220f7ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:52.808938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9548ba2-3770-4179-a133-486b220f7ed9 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:29:52.857292 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:29:52.883777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bcdb1-14d7-4fd5-81bf-20ed80c9542b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:29:52.884372 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bcdb1-14d7-4fd5-81bf-20ed80c9542b service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:29:52.884648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bcdb1-14d7-4fd5-81bf-20ed80c9542b service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:29:52.884943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bcdb1-14d7-4fd5-81bf-20ed80c9542b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:29:52.886537 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b61bcdb1-14d7-4fd5-81bf-20ed80c9542b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:29:52.994108 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bcdb1-14d7-4fd5-81bf-20ed80c9542b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:29:52.995545 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bcdb1-14d7-4fd5-81bf-20ed80c9542b service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:29:53.013567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 733/1470] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:29:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:30:02.665355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=22 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:30:03.011243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=66 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:30:12.659928 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f47a96-7d13-4d0d-8f5f-e37a0725cedf None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:12.660491 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f47a96-7d13-4d0d-8f5f-e37a0725cedf None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:12.660491 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f47a96-7d13-4d0d-8f5f-e37a0725cedf None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:12.661862 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 738/1471] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:30:12.666958 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b28fc-7ff0-4c48-98de-7c922796882f None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:12.667481 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b28fc-7ff0-4c48-98de-7c922796882f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:12.667591 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b28fc-7ff0-4c48-98de-7c922796882f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:12.734364 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-ea0b28fc-7ff0-4c48-98de-7c922796882f None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:30:12.735571 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ea0b28fc-7ff0-4c48-98de-7c922796882f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:30:12.831843 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 734/1472] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:30:12.913715 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11541e20-a93a-4683-9787-3ab11e9ea857 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:12.913945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11541e20-a93a-4683-9787-3ab11e9ea857 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:12.914220 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11541e20-a93a-4683-9787-3ab11e9ea857 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:12.915627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 739/1473] 10.222.0.74 () {54 vars in 966 bytes} [Wed Jul 27 16:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:30:12.920480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1194c-1f10-4324-a8b0-adc6aa206a48 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:12.920775 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1194c-1f10-4324-a8b0-adc6aa206a48 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:12.921096 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1194c-1f10-4324-a8b0-adc6aa206a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:12.996669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-def1194c-1f10-4324-a8b0-adc6aa206a48 None None] MFA Rules not processed for user `c50c2d1781b04a5fa8be81bce9205747`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:30:12.997915 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-def1194c-1f10-4324-a8b0-adc6aa206a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:30:13.018947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:30:13.075349 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 735/1474] 10.222.0.74 () {58 vars in 1063 bytes} [Wed Jul 27 16:30:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:30:13.083313 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11541e20-a93a-4683-9787-3ab11e9ea857 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:30:13.084857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-11541e20-a93a-4683-9787-3ab11e9ea857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:30:13.186674 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11541e20-a93a-4683-9787-3ab11e9ea857 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:30:13.257747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc3e54-6fe3-4020-a313-7037dc83de45 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c50c2d1781b04a5fa8be81bce9205747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:30:13.258355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc3e54-6fe3-4020-a313-7037dc83de45 service glance] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:13.258632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc3e54-6fe3-4020-a313-7037dc83de45 service glance] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:13.258902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc3e54-6fe3-4020-a313-7037dc83de45 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:13.260498 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-89bc3e54-6fe3-4020-a313-7037dc83de45 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:30:13.312957 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:30:13.367370 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc3e54-6fe3-4020-a313-7037dc83de45 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:30:13.368802 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc3e54-6fe3-4020-a313-7037dc83de45 service glance] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:30:13.387517 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 740/1475] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:30:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:30:15.534875 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ec45e-c17d-4053-82c5-765bb8b85fec None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:15.535436 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ec45e-c17d-4053-82c5-765bb8b85fec None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:15.535436 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ec45e-c17d-4053-82c5-765bb8b85fec None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:15.536769 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 736/1476] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:30:15.542159 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb346c5-3b9e-4a7f-accf-559b20c1e02b None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:15.542590 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb346c5-3b9e-4a7f-accf-559b20c1e02b None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:15.542745 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb346c5-3b9e-4a7f-accf-559b20c1e02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:15.608084 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-fbb346c5-3b9e-4a7f-accf-559b20c1e02b None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:30:15.609305 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-fbb346c5-3b9e-4a7f-accf-559b20c1e02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:30:15.709020 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 741/1477] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:30:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:30:15.731642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ec45e-c17d-4053-82c5-765bb8b85fec None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:30:15.733214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6e7ec45e-c17d-4053-82c5-765bb8b85fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:30:15.856654 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ec45e-c17d-4053-82c5-765bb8b85fec None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:30:15.904393 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:30:15.948967 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c83165-65fa-4a0d-83e9-7c268f352e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:30:15.949587 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c83165-65fa-4a0d-83e9-7c268f352e2a admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:15.949868 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c83165-65fa-4a0d-83e9-7c268f352e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:15.950142 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c83165-65fa-4a0d-83e9-7c268f352e2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:15.950836 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c83165-65fa-4a0d-83e9-7c268f352e2a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:30:15.952613 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: WARNING py.warnings [None req-e0c83165-65fa-4a0d-83e9-7c268f352e2a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 16:30:15.952613 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: warnings.warn(msg) Jul 27 16:30:15.952613 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: Jul 27 16:30:15.953807 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c83165-65fa-4a0d-83e9-7c268f352e2a admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:30:15.973436 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 737/1478] 10.222.0.74 () {56 vars in 1108 bytes} [Wed Jul 27 16:30:15 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:30:18.004927 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5721141-0cb8-4a3e-91be-b2a8b3a6ddef None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:18.005495 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5721141-0cb8-4a3e-91be-b2a8b3a6ddef None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:18.005495 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5721141-0cb8-4a3e-91be-b2a8b3a6ddef None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:18.006851 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 742/1479] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:30:18.012175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9582a8a-88b0-41df-827a-6c4413bdb726 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:18.012586 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9582a8a-88b0-41df-827a-6c4413bdb726 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:18.012763 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9582a8a-88b0-41df-827a-6c4413bdb726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:18.077464 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-d9582a8a-88b0-41df-827a-6c4413bdb726 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:30:18.079368 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d9582a8a-88b0-41df-827a-6c4413bdb726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:30:18.179662 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 738/1480] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:30:18.223486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5721141-0cb8-4a3e-91be-b2a8b3a6ddef None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:30:18.225028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e5721141-0cb8-4a3e-91be-b2a8b3a6ddef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:30:18.234631 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:30:18.367353 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5721141-0cb8-4a3e-91be-b2a8b3a6ddef None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:30:18.453778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:30:18.487551 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93790f4-9ed4-4b53-a5f8-8e9591cd6b95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:30:18.488163 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93790f4-9ed4-4b53-a5f8-8e9591cd6b95 service nova] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:18.488437 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93790f4-9ed4-4b53-a5f8-8e9591cd6b95 service nova] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:18.488701 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93790f4-9ed4-4b53-a5f8-8e9591cd6b95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:18.490330 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d93790f4-9ed4-4b53-a5f8-8e9591cd6b95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:30:18.578875 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93790f4-9ed4-4b53-a5f8-8e9591cd6b95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:30:18.581061 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93790f4-9ed4-4b53-a5f8-8e9591cd6b95 service nova] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:30:18.598808 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 743/1481] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:30:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:30:20.761461 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0bdb7-314f-4f73-a8b8-bb3bd842d4e7 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:20.761999 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0bdb7-314f-4f73-a8b8-bb3bd842d4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:20.761999 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0bdb7-314f-4f73-a8b8-bb3bd842d4e7 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:20.763379 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 739/1482] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:30:20.768564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a3d22-6307-4bb3-95a5-8809d00dc839 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:20.768973 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a3d22-6307-4bb3-95a5-8809d00dc839 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:20.769148 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a3d22-6307-4bb3-95a5-8809d00dc839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:20.834700 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-7d9a3d22-6307-4bb3-95a5-8809d00dc839 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:30:20.835915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7d9a3d22-6307-4bb3-95a5-8809d00dc839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:30:20.881740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:30:20.933125 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 744/1483] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:30:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:30:20.968834 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0bdb7-314f-4f73-a8b8-bb3bd842d4e7 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:30:20.970917 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0dd0bdb7-314f-4f73-a8b8-bb3bd842d4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:30:20.994271 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:30:21.073550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0bdb7-314f-4f73-a8b8-bb3bd842d4e7 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:30:21.145111 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f41c1-c7ac-4254-a531-e64737567ae2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:30:21.145715 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f41c1-c7ac-4254-a531-e64737567ae2 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:21.145994 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f41c1-c7ac-4254-a531-e64737567ae2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:21.146267 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f41c1-c7ac-4254-a531-e64737567ae2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:21.147841 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-5f5f41c1-c7ac-4254-a531-e64737567ae2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:30:21.221372 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:30:21.255451 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f41c1-c7ac-4254-a531-e64737567ae2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:30:21.256880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f41c1-c7ac-4254-a531-e64737567ae2 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:30:21.275881 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 740/1484] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:30:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:30:23.321826 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcbfa4-30e8-4229-87f6-5d3d6ab0feba None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:23.322375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcbfa4-30e8-4229-87f6-5d3d6ab0feba None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:23.322375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcbfa4-30e8-4229-87f6-5d3d6ab0feba None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:23.323739 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 745/1485] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:30:23.328823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557dc7a-d0b3-4975-8086-947e6cea0036 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:23.329268 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557dc7a-d0b3-4975-8086-947e6cea0036 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:23.329893 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557dc7a-d0b3-4975-8086-947e6cea0036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:23.395789 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-6557dc7a-d0b3-4975-8086-947e6cea0036 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:30:23.397011 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6557dc7a-d0b3-4975-8086-947e6cea0036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:30:23.494177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 741/1486] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:30:23.532207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcbfa4-30e8-4229-87f6-5d3d6ab0feba None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:30:23.534383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-5ffcbfa4-30e8-4229-87f6-5d3d6ab0feba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:30:23.640547 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcbfa4-30e8-4229-87f6-5d3d6ab0feba None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:30:23.711182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:30:23.713211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc776457-26b5-40ad-83ff-4193f1e3f79f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:30:23.713874 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc776457-26b5-40ad-83ff-4193f1e3f79f service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:23.714372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc776457-26b5-40ad-83ff-4193f1e3f79f service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:23.714858 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc776457-26b5-40ad-83ff-4193f1e3f79f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:23.716687 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-fc776457-26b5-40ad-83ff-4193f1e3f79f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:30:23.825198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc776457-26b5-40ad-83ff-4193f1e3f79f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:30:23.826572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc776457-26b5-40ad-83ff-4193f1e3f79f service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:30:23.847039 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 746/1487] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:30:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:30:25.928716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53fe6b-aa6b-429d-9657-d1be19346656 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:25.929320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53fe6b-aa6b-429d-9657-d1be19346656 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:25.929320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53fe6b-aa6b-429d-9657-d1be19346656 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:25.930655 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 742/1488] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:30:25.935777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c7f16-0cbc-42fb-886a-9c9d6502d43d None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:25.936204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c7f16-0cbc-42fb-886a-9c9d6502d43d None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:25.936357 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c7f16-0cbc-42fb-886a-9c9d6502d43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:26.000582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-bd4c7f16-0cbc-42fb-886a-9c9d6502d43d None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:30:26.001830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-bd4c7f16-0cbc-42fb-886a-9c9d6502d43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:30:26.053245 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:30:26.097817 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 747/1489] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:30:26.137901 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53fe6b-aa6b-429d-9657-d1be19346656 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:30:26.139460 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0c53fe6b-aa6b-429d-9657-d1be19346656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:30:26.175768 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:30:26.243142 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53fe6b-aa6b-429d-9657-d1be19346656 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:30:26.313824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53366580-3eb1-4fc5-8981-80aa159d20c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:30:26.314441 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53366580-3eb1-4fc5-8981-80aa159d20c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:26.314723 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53366580-3eb1-4fc5-8981-80aa159d20c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:26.314999 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53366580-3eb1-4fc5-8981-80aa159d20c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:26.316625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-53366580-3eb1-4fc5-8981-80aa159d20c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:30:26.396331 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:30:26.425334 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53366580-3eb1-4fc5-8981-80aa159d20c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:30:26.426764 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53366580-3eb1-4fc5-8981-80aa159d20c5 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:30:26.444113 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 743/1490] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:30:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:30:28.644171 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ab868-c8bf-45c3-9f25-fabbb7e85bc3 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:28.644801 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ab868-c8bf-45c3-9f25-fabbb7e85bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:28.644801 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ab868-c8bf-45c3-9f25-fabbb7e85bc3 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:28.646109 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 748/1491] 10.222.0.74 () {54 vars in 926 bytes} [Wed Jul 27 16:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:30:28.651373 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b842e5d-f519-42aa-a173-f889e93d8db2 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:28.651820 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b842e5d-f519-42aa-a173-f889e93d8db2 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:28.651929 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b842e5d-f519-42aa-a173-f889e93d8db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:28.719349 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-7b842e5d-f519-42aa-a173-f889e93d8db2 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:30:28.721186 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-7b842e5d-f519-42aa-a173-f889e93d8db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:30:28.818188 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 744/1492] 10.222.0.74 () {58 vars in 1023 bytes} [Wed Jul 27 16:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:30:28.858177 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ab868-c8bf-45c3-9f25-fabbb7e85bc3 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:30:28.859712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-317ab868-c8bf-45c3-9f25-fabbb7e85bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:30:28.963477 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ab868-c8bf-45c3-9f25-fabbb7e85bc3 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:30:29.035801 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34d99c-289e-4e1e-ab66-4502b029e40b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:30:29.036411 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34d99c-289e-4e1e-ab66-4502b029e40b service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:30:29.036691 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34d99c-289e-4e1e-ab66-4502b029e40b service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:30:29.036992 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34d99c-289e-4e1e-ab66-4502b029e40b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:30:29.038597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-3e34d99c-289e-4e1e-ab66-4502b029e40b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:30:29.047494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:30:29.145766 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34d99c-289e-4e1e-ab66-4502b029e40b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:30:29.147451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34d99c-289e-4e1e-ab66-4502b029e40b service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:30:29.166171 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 749/1493] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:30:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:30:38.816364 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=82 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:30:39.163883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=50 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:31:56.496139 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c7e94-6807-478c-bf8d-07a4d972021f None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:31:56.496626 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c7e94-6807-478c-bf8d-07a4d972021f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:31:56.496730 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c7e94-6807-478c-bf8d-07a4d972021f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:31:56.570940 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-b60c7e94-6807-478c-bf8d-07a4d972021f None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:31:56.572149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b60c7e94-6807-478c-bf8d-07a4d972021f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:31:56.666419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 745/1494] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:31:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:31:56.674879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef181252-1c30-418e-8670-d7440ddb445f None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:31:56.675365 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef181252-1c30-418e-8670-d7440ddb445f None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:31:56.675365 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef181252-1c30-418e-8670-d7440ddb445f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:31:56.749024 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-ef181252-1c30-418e-8670-d7440ddb445f None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:31:56.750712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-ef181252-1c30-418e-8670-d7440ddb445f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:31:56.844224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 750/1495] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:31:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:31:56.848954 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c7e94-6807-478c-bf8d-07a4d972021f None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:31:56.850203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b60c7e94-6807-478c-bf8d-07a4d972021f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:31:56.861687 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c7e94-6807-478c-bf8d-07a4d972021f None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:31:56.922581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:31:56.957256 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd33cb-f581-4f40-b256-3653d65bc92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:31:56.957881 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd33cb-f581-4f40-b256-3653d65bc92e admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:31:56.958431 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd33cb-f581-4f40-b256-3653d65bc92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:31:56.958705 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd33cb-f581-4f40-b256-3653d65bc92e admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:31:56.959403 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd33cb-f581-4f40-b256-3653d65bc92e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:31:56.959709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd33cb-f581-4f40-b256-3653d65bc92e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:31:56.960729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: WARNING py.warnings [None req-d6fd33cb-f581-4f40-b256-3653d65bc92e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 16:31:56.960729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: warnings.warn(msg) Jul 27 16:31:56.960729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: Jul 27 16:31:56.961643 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd33cb-f581-4f40-b256-3653d65bc92e admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:31:56.973390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 746/1496] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:31:56.981721 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef181252-1c30-418e-8670-d7440ddb445f None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:31:56.991399 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef181252-1c30-418e-8670-d7440ddb445f None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:31:57.051944 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:31:57.086108 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd720b-42b6-4a4b-b9c2-7d4502bd4ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:31:57.086720 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd720b-42b6-4a4b-b9c2-7d4502bd4ebe admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:31:57.086996 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd720b-42b6-4a4b-b9c2-7d4502bd4ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:31:57.087264 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd720b-42b6-4a4b-b9c2-7d4502bd4ebe admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:31:57.087922 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd720b-42b6-4a4b-b9c2-7d4502bd4ebe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:31:57.089690 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: WARNING py.warnings [None req-89fd720b-42b6-4a4b-b9c2-7d4502bd4ebe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 16:31:57.089690 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: warnings.warn(msg) Jul 27 16:31:57.089690 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: Jul 27 16:31:57.090872 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd720b-42b6-4a4b-b9c2-7d4502bd4ebe admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:31:57.146522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 751/1497] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:31:56 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:31:57.150809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd33cb-f581-4f40-b256-3653d65bc92e admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:31:57.152342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d6fd33cb-f581-4f40-b256-3653d65bc92e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:31:57.186101 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd33cb-f581-4f40-b256-3653d65bc92e admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:31:57.281749 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33a80b-f673-4812-b3d1-930728341efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:31:57.282296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33a80b-f673-4812-b3d1-930728341efe admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:31:57.282578 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33a80b-f673-4812-b3d1-930728341efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:31:57.282854 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33a80b-f673-4812-b3d1-930728341efe admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:31:57.283518 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33a80b-f673-4812-b3d1-930728341efe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:31:57.284539 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: WARNING py.warnings [None req-ab33a80b-f673-4812-b3d1-930728341efe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 16:31:57.284539 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: warnings.warn(msg) Jul 27 16:31:57.284539 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: Jul 27 16:31:57.285779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33a80b-f673-4812-b3d1-930728341efe admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:31:57.334000 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 747/1498] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:31:57 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:31:57.338068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd720b-42b6-4a4b-b9c2-7d4502bd4ebe admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:31:57.339569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-89fd720b-42b6-4a4b-b9c2-7d4502bd4ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:31:57.409283 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd720b-42b6-4a4b-b9c2-7d4502bd4ebe admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:31:57.503041 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:31:57.515865 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903a367-ec16-451f-af3b-45cda8b980eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:31:57.516883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903a367-ec16-451f-af3b-45cda8b980eb admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:31:57.517417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903a367-ec16-451f-af3b-45cda8b980eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:31:57.517761 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903a367-ec16-451f-af3b-45cda8b980eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:31:57.519107 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903a367-ec16-451f-af3b-45cda8b980eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:31:57.520125 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: WARNING py.warnings [None req-6903a367-ec16-451f-af3b-45cda8b980eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 16:31:57.520125 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: warnings.warn(msg) Jul 27 16:31:57.520125 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: Jul 27 16:31:57.521045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903a367-ec16-451f-af3b-45cda8b980eb admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:31:57.534258 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 752/1499] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:31:57.538382 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33a80b-f673-4812-b3d1-930728341efe admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:31:57.539905 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ab33a80b-f673-4812-b3d1-930728341efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:31:57.551939 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33a80b-f673-4812-b3d1-930728341efe admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:31:57.577451 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:31:57.777953 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47418f1c-5e90-4bd0-979b-d2f423f63c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:31:57.778648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47418f1c-5e90-4bd0-979b-d2f423f63c7e admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:31:57.778929 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47418f1c-5e90-4bd0-979b-d2f423f63c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:31:57.779199 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47418f1c-5e90-4bd0-979b-d2f423f63c7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:31:57.779902 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47418f1c-5e90-4bd0-979b-d2f423f63c7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:31:57.781662 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: WARNING py.warnings [None req-47418f1c-5e90-4bd0-979b-d2f423f63c7e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 16:31:57.781662 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: warnings.warn(msg) Jul 27 16:31:57.781662 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: Jul 27 16:31:57.782552 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47418f1c-5e90-4bd0-979b-d2f423f63c7e admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:31:57.795293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 748/1500] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:31:57.799287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903a367-ec16-451f-af3b-45cda8b980eb admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:31:57.800828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-6903a367-ec16-451f-af3b-45cda8b980eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:31:57.834794 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903a367-ec16-451f-af3b-45cda8b980eb admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:31:57.928602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ce588-f461-4617-b98d-5641fb892801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:31:57.929301 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ce588-f461-4617-b98d-5641fb892801 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:31:57.929597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ce588-f461-4617-b98d-5641fb892801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:31:57.929864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ce588-f461-4617-b98d-5641fb892801 admin admin] PATH_INFO: `/v3/projects/4a5801a9597048d4a1e97c3a91cc5f2e/users/09bd68c6c42342cb8b244632e75a64a1/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:31:57.952285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ce588-f461-4617-b98d-5641fb892801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a5801a9597048d4a1e97c3a91cc5f2e, user_id=09bd68c6c42342cb8b244632e75a64a1, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:31:57.953987 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: WARNING py.warnings [None req-970ce588-f461-4617-b98d-5641fb892801 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 16:31:57.953987 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: warnings.warn(msg) Jul 27 16:31:57.953987 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: Jul 27 16:31:57.956032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ce588-f461-4617-b98d-5641fb892801 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:31:57.981194 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 753/1501] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:31:57 2022] PUT /identity/v3/projects/4a5801a9597048d4a1e97c3a91cc5f2e/users/09bd68c6c42342cb8b244632e75a64a1/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:31:57.993358 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f6958-fd97-450c-8b33-d514f4aa141f None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:31:57.993645 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f6958-fd97-450c-8b33-d514f4aa141f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:31:57.993942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f6958-fd97-450c-8b33-d514f4aa141f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:31:58.052787 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-815f6958-fd97-450c-8b33-d514f4aa141f None None] MFA Rules not processed for user `09bd68c6c42342cb8b244632e75a64a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:31:58.054048 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-815f6958-fd97-450c-8b33-d514f4aa141f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:31:58.133064 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 749/1502] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:31:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:31:58.146882 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32d69c-d708-467f-b3dc-a2544ab53ac3 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:31:58.147175 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32d69c-d708-467f-b3dc-a2544ab53ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:31:58.147457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32d69c-d708-467f-b3dc-a2544ab53ac3 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:31:58.149064 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 754/1503] 10.222.0.74 () {54 vars in 964 bytes} [Wed Jul 27 16:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:31:58.153589 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c977cf-699e-49cd-b9ca-159ceeeaf057 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:31:58.153879 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c977cf-699e-49cd-b9ca-159ceeeaf057 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:31:58.154156 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c977cf-699e-49cd-b9ca-159ceeeaf057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:31:58.184502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:31:58.229211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-78c977cf-699e-49cd-b9ca-159ceeeaf057 None None] MFA Rules not processed for user `dd352d423bff418f932b434c51b15d23`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:31:58.230419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-78c977cf-699e-49cd-b9ca-159ceeeaf057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:31:58.305317 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 750/1504] 10.222.0.74 () {58 vars in 1061 bytes} [Wed Jul 27 16:31:58 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:31:58.313469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32d69c-d708-467f-b3dc-a2544ab53ac3 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:31:58.314983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-9d32d69c-d708-467f-b3dc-a2544ab53ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:31:58.365380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:31:58.420027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32d69c-d708-467f-b3dc-a2544ab53ac3 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:31:58.493386 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36025451-5bf7-45b3-a410-74a544ec2600 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:31:58.494004 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36025451-5bf7-45b3-a410-74a544ec2600 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:31:58.494289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36025451-5bf7-45b3-a410-74a544ec2600 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:31:58.494568 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36025451-5bf7-45b3-a410-74a544ec2600 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:31:58.496185 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-36025451-5bf7-45b3-a410-74a544ec2600 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:31:58.600177 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36025451-5bf7-45b3-a410-74a544ec2600 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:31:58.601614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36025451-5bf7-45b3-a410-74a544ec2600 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:31:58.605160 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:31:58.620426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 755/1505] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:31:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:31:58.667611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51374f-c776-466a-b223-9f4740625988 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:31:58.668098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51374f-c776-466a-b223-9f4740625988 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:31:58.668189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51374f-c776-466a-b223-9f4740625988 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:31:58.669546 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 751/1506] 10.222.0.74 () {52 vars in 830 bytes} [Wed Jul 27 16:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:31:58.673768 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36025451-5bf7-45b3-a410-74a544ec2600 service cinder] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:31:58.674985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-36025451-5bf7-45b3-a410-74a544ec2600 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:31:58.774650 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36025451-5bf7-45b3-a410-74a544ec2600 service cinder] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:31:58.866540 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdafdd8-6d16-44f2-8233-aac240e891c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:31:58.867187 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdafdd8-6d16-44f2-8233-aac240e891c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:31:58.867359 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdafdd8-6d16-44f2-8233-aac240e891c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:31:58.867632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdafdd8-6d16-44f2-8233-aac240e891c7 admin admin] PATH_INFO: `/v3/users/09bd68c6c42342cb8b244632e75a64a1` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:31:58.880600 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdafdd8-6d16-44f2-8233-aac240e891c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09bd68c6c42342cb8b244632e75a64a1)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:31:58.881660 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: WARNING py.warnings [None req-ebdafdd8-6d16-44f2-8233-aac240e891c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 16:31:58.881660 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: warnings.warn(msg) Jul 27 16:31:58.881660 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: Jul 27 16:31:58.882881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdafdd8-6d16-44f2-8233-aac240e891c7 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:31:58.892325 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:31:58.946319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ebdafdd8-6d16-44f2-8233-aac240e891c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09bd68c6c42342cb8b244632e75a64a1'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:31:58.955065 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ebdafdd8-6d16-44f2-8233-aac240e891c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09bd68c6c42342cb8b244632e75a64a1'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:31:58.973247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ebdafdd8-6d16-44f2-8233-aac240e891c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09bd68c6c42342cb8b244632e75a64a1'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:31:58.985791 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ebdafdd8-6d16-44f2-8233-aac240e891c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09bd68c6c42342cb8b244632e75a64a1'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:31:58.986235 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ebdafdd8-6d16-44f2-8233-aac240e891c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09bd68c6c42342cb8b244632e75a64a1'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:31:58.991770 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 756/1507] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:31:58 2022] DELETE /identity/v3/users/09bd68c6c42342cb8b244632e75a64a1 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:31:58.996151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aba6f8-663a-478e-9577-02b35b89615b None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:31:58.996430 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aba6f8-663a-478e-9577-02b35b89615b None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:31:58.996700 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aba6f8-663a-478e-9577-02b35b89615b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:31:59.052255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-f5aba6f8-663a-478e-9577-02b35b89615b None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:31:59.053463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-f5aba6f8-663a-478e-9577-02b35b89615b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:31:59.129531 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:31:59.142524 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 752/1508] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:31:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:31:59.152097 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdafdd8-6d16-44f2-8233-aac240e891c7 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:31:59.153408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-ebdafdd8-6d16-44f2-8233-aac240e891c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:31:59.254350 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdafdd8-6d16-44f2-8233-aac240e891c7 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:31:59.307307 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:31:59.327305 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed4d128-7809-4107-b110-0b2d7f634a83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:31:59.328433 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4d128-7809-4107-b110-0b2d7f634a83 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:31:59.328713 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4d128-7809-4107-b110-0b2d7f634a83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:31:59.329016 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4d128-7809-4107-b110-0b2d7f634a83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:31:59.330623 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-4ed4d128-7809-4107-b110-0b2d7f634a83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:31:59.436340 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed4d128-7809-4107-b110-0b2d7f634a83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:31:59.437749 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed4d128-7809-4107-b110-0b2d7f634a83 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:31:59.456160 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 757/1509] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:31:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:31:59.801669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aba6f8-663a-478e-9577-02b35b89615b None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:31:59.802896 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-f5aba6f8-663a-478e-9577-02b35b89615b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:31:59.814993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aba6f8-663a-478e-9577-02b35b89615b None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:31:59.913038 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00176d0d-8c9d-4dcb-8100-a8ea884ef533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:31:59.913683 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00176d0d-8c9d-4dcb-8100-a8ea884ef533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:31:59.913964 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00176d0d-8c9d-4dcb-8100-a8ea884ef533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:31:59.914238 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00176d0d-8c9d-4dcb-8100-a8ea884ef533 admin admin] PATH_INFO: `/v3/projects/4a5801a9597048d4a1e97c3a91cc5f2e` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:31:59.924372 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00176d0d-8c9d-4dcb-8100-a8ea884ef533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a5801a9597048d4a1e97c3a91cc5f2e)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:31:59.925435 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: WARNING py.warnings [None req-00176d0d-8c9d-4dcb-8100-a8ea884ef533 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 16:31:59.925435 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: warnings.warn(msg) Jul 27 16:31:59.925435 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: Jul 27 16:31:59.926638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00176d0d-8c9d-4dcb-8100-a8ea884ef533 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:31:59.987232 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-00176d0d-8c9d-4dcb-8100-a8ea884ef533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a5801a9597048d4a1e97c3a91cc5f2e', 'request_id': 'req-00176d0d-8c9d-4dcb-8100-a8ea884ef533'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:31:59.993285 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-00176d0d-8c9d-4dcb-8100-a8ea884ef533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a5801a9597048d4a1e97c3a91cc5f2e', 'request_id': 'req-00176d0d-8c9d-4dcb-8100-a8ea884ef533'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:32:00.005296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-00176d0d-8c9d-4dcb-8100-a8ea884ef533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a5801a9597048d4a1e97c3a91cc5f2e', 'request_id': 'req-00176d0d-8c9d-4dcb-8100-a8ea884ef533'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:32:00.034216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-00176d0d-8c9d-4dcb-8100-a8ea884ef533 admin admin] The token cache is being invalidate because project 4a5801a9597048d4a1e97c3a91cc5f2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:32:00.034501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-00176d0d-8c9d-4dcb-8100-a8ea884ef533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:32:00.034922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-00176d0d-8c9d-4dcb-8100-a8ea884ef533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:32:00.036337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 753/1510] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:31:59 2022] DELETE /identity/v3/projects/4a5801a9597048d4a1e97c3a91cc5f2e => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:32:09.453880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=62 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:32:10.032097 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:33:42.188969 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15826a-4e00-41bf-a153-b0799d795520 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:33:42.189491 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15826a-4e00-41bf-a153-b0799d795520 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:33:42.189595 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15826a-4e00-41bf-a153-b0799d795520 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:33:42.190882 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 16:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:33:42.195624 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6cc5c-a2c2-4e8e-9fbb-510fd22435e3 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:33:42.196016 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6cc5c-a2c2-4e8e-9fbb-510fd22435e3 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:33:42.196190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6cc5c-a2c2-4e8e-9fbb-510fd22435e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:33:42.278348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-3bb6cc5c-a2c2-4e8e-9fbb-510fd22435e3 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:33:42.279529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-3bb6cc5c-a2c2-4e8e-9fbb-510fd22435e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:33:42.382200 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 16:33:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:33:42.534894 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f15826a-4e00-41bf-a153-b0799d795520 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:33:42.537017 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7f15826a-4e00-41bf-a153-b0799d795520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:33:42.653029 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f15826a-4e00-41bf-a153-b0799d795520 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:33:42.658612 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c03f6f-af17-4b45-9838-198866ff994d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:33:42.659226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c03f6f-af17-4b45-9838-198866ff994d service nova] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:33:42.659508 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c03f6f-af17-4b45-9838-198866ff994d service nova] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:33:42.659784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c03f6f-af17-4b45-9838-198866ff994d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:33:42.667720 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c03f6f-af17-4b45-9838-198866ff994d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:33:42.669327 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c03f6f-af17-4b45-9838-198866ff994d service nova] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:33:42.679245 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 759/1513] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:33:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:33:43.508649 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61fd48-3321-4351-8073-5304a073a422 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:33:43.509241 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61fd48-3321-4351-8073-5304a073a422 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:33:43.509406 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61fd48-3321-4351-8073-5304a073a422 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:33:43.510809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 16:33:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:33:43.515681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bbe99-baa2-4e32-a1d5-3deb20632474 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:33:43.516124 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bbe99-baa2-4e32-a1d5-3deb20632474 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:33:43.516228 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bbe99-baa2-4e32-a1d5-3deb20632474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:33:43.535732 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-293bbe99-baa2-4e32-a1d5-3deb20632474 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:33:43.536976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-293bbe99-baa2-4e32-a1d5-3deb20632474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:33:43.545379 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 16:33:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:33:43.698979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca61fd48-3321-4351-8073-5304a073a422 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:33:43.713838 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca61fd48-3321-4351-8073-5304a073a422 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:33:43.719210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab61837-9faa-421e-a17d-2f5d49b1b2cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:33:43.719811 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab61837-9faa-421e-a17d-2f5d49b1b2cf service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:33:43.720088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab61837-9faa-421e-a17d-2f5d49b1b2cf service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:33:43.720359 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab61837-9faa-421e-a17d-2f5d49b1b2cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:33:43.728235 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab61837-9faa-421e-a17d-2f5d49b1b2cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:33:43.729826 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab61837-9faa-421e-a17d-2f5d49b1b2cf service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:33:43.739589 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 756/1516] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:33:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:33:44.889594 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dba6c4-7224-490f-9ea4-68c42a31d668 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:33:44.890129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dba6c4-7224-490f-9ea4-68c42a31d668 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:33:44.890129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dba6c4-7224-490f-9ea4-68c42a31d668 None None] PATH_INFO: `/v3` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:33:44.891345 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 761/1517] 10.222.0.20 () {54 vars in 925 bytes} [Wed Jul 27 16:33:44 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:33:44.899983 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84826236-0b96-4eba-9d28-6f73a5af1ef0 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:33:44.900363 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84826236-0b96-4eba-9d28-6f73a5af1ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:33:44.900530 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84826236-0b96-4eba-9d28-6f73a5af1ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:33:44.958363 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-84826236-0b96-4eba-9d28-6f73a5af1ef0 None None] MFA Rules not processed for user `91f8e0f82a8541aead6cd8cc45f4a86b`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:33:44.959566 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-84826236-0b96-4eba-9d28-6f73a5af1ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:33:44.975994 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:33:45.029370 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 757/1518] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Jul 27 16:33:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:33:49.003159 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fa97d-11b4-4473-984e-f8d4941aa658 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:33:49.003698 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fa97d-11b4-4473-984e-f8d4941aa658 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:33:49.003698 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fa97d-11b4-4473-984e-f8d4941aa658 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:33:49.005270 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 762/1519] 10.222.0.74 () {54 vars in 961 bytes} [Wed Jul 27 16:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:33:49.009963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509973e3-101d-484b-906f-a0479c40337d None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:33:49.010381 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509973e3-101d-484b-906f-a0479c40337d None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:33:49.010628 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509973e3-101d-484b-906f-a0479c40337d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:33:49.031233 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-509973e3-101d-484b-906f-a0479c40337d None None] MFA Rules not processed for user `91f8e0f82a8541aead6cd8cc45f4a86b`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:33:49.033063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-509973e3-101d-484b-906f-a0479c40337d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:33:49.041994 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 758/1520] 10.222.0.74 () {58 vars in 1058 bytes} [Wed Jul 27 16:33:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:33:49.050180 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fa97d-11b4-4473-984e-f8d4941aa658 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:33:49.067911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fa97d-11b4-4473-984e-f8d4941aa658 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:33:49.073104 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78855272-4831-478b-aec0-c99106da94db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f8e0f82a8541aead6cd8cc45f4a86b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:33:49.073714 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78855272-4831-478b-aec0-c99106da94db service placement] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:33:49.074051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78855272-4831-478b-aec0-c99106da94db service placement] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:33:49.074329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78855272-4831-478b-aec0-c99106da94db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:33:49.083331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78855272-4831-478b-aec0-c99106da94db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:33:49.084935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78855272-4831-478b-aec0-c99106da94db service placement] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:33:49.095817 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 763/1521] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:33:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:33:54.989757 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5e573-efc6-4bf0-9e2f-0cd66a7c095f None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:33:54.990338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5e573-efc6-4bf0-9e2f-0cd66a7c095f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:33:54.990338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5e573-efc6-4bf0-9e2f-0cd66a7c095f None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:33:54.991677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 16:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:33:54.996398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599e51f-2658-4b9f-9aa2-72c1beacc1f7 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:33:54.996788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599e51f-2658-4b9f-9aa2-72c1beacc1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:33:54.996989 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599e51f-2658-4b9f-9aa2-72c1beacc1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:33:55.016077 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-b599e51f-2658-4b9f-9aa2-72c1beacc1f7 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:33:55.017843 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b599e51f-2658-4b9f-9aa2-72c1beacc1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:33:55.025678 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 16:33:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:33:55.178919 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5e573-efc6-4bf0-9e2f-0cd66a7c095f None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:33:55.194091 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5e573-efc6-4bf0-9e2f-0cd66a7c095f None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:33:55.199586 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47455b-db00-44fe-9656-1712506e8dd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:33:55.200192 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47455b-db00-44fe-9656-1712506e8dd5 service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:33:55.200478 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47455b-db00-44fe-9656-1712506e8dd5 service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:33:55.200757 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47455b-db00-44fe-9656-1712506e8dd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:33:55.209534 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f47455b-db00-44fe-9656-1712506e8dd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:33:55.211454 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f47455b-db00-44fe-9656-1712506e8dd5 service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:33:55.220701 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 760/1524] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:33:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:33:56.033985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b60e23-4c7d-482c-9d6c-46427a911273 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:33:56.034548 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b60e23-4c7d-482c-9d6c-46427a911273 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:33:56.034548 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b60e23-4c7d-482c-9d6c-46427a911273 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:33:56.035892 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 16:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:33:56.040595 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32ddc5-7a17-405a-b2c6-7425f8564902 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:33:56.041063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32ddc5-7a17-405a-b2c6-7425f8564902 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:33:56.041162 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32ddc5-7a17-405a-b2c6-7425f8564902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:33:56.060185 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-8e32ddc5-7a17-405a-b2c6-7425f8564902 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:33:56.061453 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-8e32ddc5-7a17-405a-b2c6-7425f8564902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:33:56.070102 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 16:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:33:56.221481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b60e23-4c7d-482c-9d6c-46427a911273 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:33:56.236265 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b60e23-4c7d-482c-9d6c-46427a911273 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:33:56.241691 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827b5c7-6864-4c1b-be22-5951ff25aacc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:33:56.242287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827b5c7-6864-4c1b-be22-5951ff25aacc service nova] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:33:56.242567 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827b5c7-6864-4c1b-be22-5951ff25aacc service nova] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:33:56.242841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827b5c7-6864-4c1b-be22-5951ff25aacc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:33:56.251400 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827b5c7-6864-4c1b-be22-5951ff25aacc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:33:56.253006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827b5c7-6864-4c1b-be22-5951ff25aacc service nova] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:33:56.262239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 766/1527] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:33:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:34:00.371818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e986af3-06bd-4b46-8099-634254fdc522 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:34:00.372313 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e986af3-06bd-4b46-8099-634254fdc522 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:34:00.372418 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e986af3-06bd-4b46-8099-634254fdc522 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:34:00.373754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 16:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:34:00.379985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8c3de-f710-42fb-aba9-160f93b789fe None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:34:00.380455 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8c3de-f710-42fb-aba9-160f93b789fe None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:34:00.380870 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8c3de-f710-42fb-aba9-160f93b789fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:34:00.403500 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-87a8c3de-f710-42fb-aba9-160f93b789fe None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:34:00.404712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-87a8c3de-f710-42fb-aba9-160f93b789fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:34:00.415239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 16:34:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:34:00.497731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e986af3-06bd-4b46-8099-634254fdc522 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:34:00.499645 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-8e986af3-06bd-4b46-8099-634254fdc522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:34:00.581566 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e986af3-06bd-4b46-8099-634254fdc522 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:34:00.586141 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf9fb7-59e3-4c26-b0c5-b1511b7fa34a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:34:00.586760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf9fb7-59e3-4c26-b0c5-b1511b7fa34a service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:34:00.587050 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf9fb7-59e3-4c26-b0c5-b1511b7fa34a service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:34:00.587329 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf9fb7-59e3-4c26-b0c5-b1511b7fa34a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:34:00.596441 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf9fb7-59e3-4c26-b0c5-b1511b7fa34a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:34:00.597913 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf9fb7-59e3-4c26-b0c5-b1511b7fa34a service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:34:00.608091 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 763/1530] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:34:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:34:02.604237 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f3245-9dad-4408-b4b9-8395de4e8a63 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:34:02.604788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f3245-9dad-4408-b4b9-8395de4e8a63 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:34:02.604788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f3245-9dad-4408-b4b9-8395de4e8a63 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:34:02.606182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 16:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:34:02.611898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9474dee-2bab-4988-9dc7-1bb8526e3752 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:34:02.612318 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9474dee-2bab-4988-9dc7-1bb8526e3752 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:34:02.612424 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9474dee-2bab-4988-9dc7-1bb8526e3752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:34:02.632194 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-d9474dee-2bab-4988-9dc7-1bb8526e3752 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:34:02.633405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d9474dee-2bab-4988-9dc7-1bb8526e3752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:34:02.641079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 16:34:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:34:02.677122 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f3245-9dad-4408-b4b9-8395de4e8a63 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:34:02.678764 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-272f3245-9dad-4408-b4b9-8395de4e8a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:34:02.692356 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f3245-9dad-4408-b4b9-8395de4e8a63 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:34:02.695893 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7395e1f-70d1-4bac-a6af-e4a1bcef0f2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:34:02.696497 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7395e1f-70d1-4bac-a6af-e4a1bcef0f2f service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:34:02.696780 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7395e1f-70d1-4bac-a6af-e4a1bcef0f2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:34:02.697077 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7395e1f-70d1-4bac-a6af-e4a1bcef0f2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:34:02.706487 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7395e1f-70d1-4bac-a6af-e4a1bcef0f2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:34:02.707899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7395e1f-70d1-4bac-a6af-e4a1bcef0f2f service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:34:02.717363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 769/1533] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:34:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:34:06.258351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f67ff5-6564-48a2-9f61-b4b56afd142e None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:34:06.258880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f67ff5-6564-48a2-9f61-b4b56afd142e None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:34:06.258987 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f67ff5-6564-48a2-9f61-b4b56afd142e None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:34:06.260256 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 16:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:34:06.265463 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3621fd3-a278-416d-a978-ae2d7ce21daa None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:34:06.265914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3621fd3-a278-416d-a978-ae2d7ce21daa None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:34:06.266020 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3621fd3-a278-416d-a978-ae2d7ce21daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:34:06.286806 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-b3621fd3-a278-416d-a978-ae2d7ce21daa None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:34:06.288064 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b3621fd3-a278-416d-a978-ae2d7ce21daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:34:06.296133 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 16:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:34:06.598114 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f67ff5-6564-48a2-9f61-b4b56afd142e None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:34:06.599788 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-a3f67ff5-6564-48a2-9f61-b4b56afd142e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:34:06.663156 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:34:06.676319 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f67ff5-6564-48a2-9f61-b4b56afd142e None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:34:06.679860 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e97bb-e4e7-4645-860e-137264bcc711 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:34:06.680467 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e97bb-e4e7-4645-860e-137264bcc711 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:34:06.680752 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e97bb-e4e7-4645-860e-137264bcc711 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:34:06.681061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e97bb-e4e7-4645-860e-137264bcc711 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:34:06.690065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e97bb-e4e7-4645-860e-137264bcc711 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:34:06.691489 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e97bb-e4e7-4645-860e-137264bcc711 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:34:06.700753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 766/1536] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:34:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:34:08.722996 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f46dc-d5f8-4024-8941-c66f67170079 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:34:08.723552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f46dc-d5f8-4024-8941-c66f67170079 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:34:08.723552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f46dc-d5f8-4024-8941-c66f67170079 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:34:08.724985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 16:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:34:08.730194 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b865ef0-1bff-48ca-8a1e-4f688d28eab3 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:34:08.730618 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b865ef0-1bff-48ca-8a1e-4f688d28eab3 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:34:08.730753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b865ef0-1bff-48ca-8a1e-4f688d28eab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:34:08.750241 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-3b865ef0-1bff-48ca-8a1e-4f688d28eab3 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:34:08.751386 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-3b865ef0-1bff-48ca-8a1e-4f688d28eab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:34:08.759042 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 16:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:34:09.083945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f46dc-d5f8-4024-8941-c66f67170079 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:34:09.085851 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-1a4f46dc-d5f8-4024-8941-c66f67170079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:34:09.099476 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f46dc-d5f8-4024-8941-c66f67170079 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:34:09.102994 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f13c3-1846-41f0-a6ed-e3795144834c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:34:09.103598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f13c3-1846-41f0-a6ed-e3795144834c service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:34:09.103878 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f13c3-1846-41f0-a6ed-e3795144834c service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:34:09.104171 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f13c3-1846-41f0-a6ed-e3795144834c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:34:09.113178 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f13c3-1846-41f0-a6ed-e3795144834c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:34:09.114599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f13c3-1846-41f0-a6ed-e3795144834c service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:34:09.123871 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 772/1539] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:34:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:34:18.745607 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=11 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:34:19.119806 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:35:41.494931 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576d45f-b742-4a3a-b5d8-1151b6a4e1b0 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:41.495599 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576d45f-b742-4a3a-b5d8-1151b6a4e1b0 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:41.495599 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576d45f-b742-4a3a-b5d8-1151b6a4e1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:41.520850 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-c576d45f-b742-4a3a-b5d8-1151b6a4e1b0 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:41.522106 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c576d45f-b742-4a3a-b5d8-1151b6a4e1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:41.533139 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 768/1540] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:35:41.542460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8c95a-add3-4b6e-9685-d53021493077 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:41.543050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8c95a-add3-4b6e-9685-d53021493077 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:41.543050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8c95a-add3-4b6e-9685-d53021493077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:41.568278 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-7ad8c95a-add3-4b6e-9685-d53021493077 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:41.569647 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7ad8c95a-add3-4b6e-9685-d53021493077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:41.583467 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 773/1541] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:41.588498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576d45f-b742-4a3a-b5d8-1151b6a4e1b0 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:41.599953 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd39576-ca9a-4b94-ba00-4c13f78f02bd None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:41.600297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd39576-ca9a-4b94-ba00-4c13f78f02bd None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:41.600508 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd39576-ca9a-4b94-ba00-4c13f78f02bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:41.611002 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576d45f-b742-4a3a-b5d8-1151b6a4e1b0 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:41.618992 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d1643-6dfa-498d-973c-809d24a462d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:41.625255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d1643-6dfa-498d-973c-809d24a462d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:41.625566 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d1643-6dfa-498d-973c-809d24a462d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:41.625891 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d1643-6dfa-498d-973c-809d24a462d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:41.628446 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d1643-6dfa-498d-973c-809d24a462d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:35:41.628757 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d1643-6dfa-498d-973c-809d24a462d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:41.630715 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d1643-6dfa-498d-973c-809d24a462d5 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:41.661153 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-cfd39576-ca9a-4b94-ba00-4c13f78f02bd None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:41.662733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-cfd39576-ca9a-4b94-ba00-4c13f78f02bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:41.666590 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 769/1542] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:35:41.667993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b25db2-4c4a-4d9e-8865-ae7c88fc7df6 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:41.668284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b25db2-4c4a-4d9e-8865-ae7c88fc7df6 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:41.668561 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b25db2-4c4a-4d9e-8865-ae7c88fc7df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:41.683535 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 774/1543] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:41.684349 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd39576-ca9a-4b94-ba00-4c13f78f02bd None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:41.695663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd39576-ca9a-4b94-ba00-4c13f78f02bd None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:41.712497 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b660d51f-7fa8-42f8-8149-d62c2623156d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:41.713166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660d51f-7fa8-42f8-8149-d62c2623156d admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:41.713447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660d51f-7fa8-42f8-8149-d62c2623156d admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:41.713722 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660d51f-7fa8-42f8-8149-d62c2623156d admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:41.714391 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b660d51f-7fa8-42f8-8149-d62c2623156d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:41.716321 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b660d51f-7fa8-42f8-8149-d62c2623156d admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:41.720106 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-08b25db2-4c4a-4d9e-8865-ae7c88fc7df6 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:41.721616 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-08b25db2-4c4a-4d9e-8865-ae7c88fc7df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:41.737951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 770/1544] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:41.739256 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286f48c-7b5a-40c6-9a26-82444e4f226d None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:41.739543 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286f48c-7b5a-40c6-9a26-82444e4f226d None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:41.739817 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286f48c-7b5a-40c6-9a26-82444e4f226d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:41.786128 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-c286f48c-7b5a-40c6-9a26-82444e4f226d None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:41.787362 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c286f48c-7b5a-40c6-9a26-82444e4f226d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:41.809129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 775/1545] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:35:41 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:35:41.810501 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376b819-42a4-4a44-b2b5-15e66277c5ee None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:41.810794 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376b819-42a4-4a44-b2b5-15e66277c5ee None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:41.811074 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376b819-42a4-4a44-b2b5-15e66277c5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:41.815317 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 771/1546] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:41.816669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06449b99-e254-4a6a-84cf-8a88ba0c7663 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:41.817560 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06449b99-e254-4a6a-84cf-8a88ba0c7663 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:41.817846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06449b99-e254-4a6a-84cf-8a88ba0c7663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:41.847003 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-06449b99-e254-4a6a-84cf-8a88ba0c7663 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:41.847264 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-6376b819-42a4-4a44-b2b5-15e66277c5ee None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:41.848785 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-6376b819-42a4-4a44-b2b5-15e66277c5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:41.849759 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-06449b99-e254-4a6a-84cf-8a88ba0c7663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:41.913432 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 776/1547] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:41.914694 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbfa9a-3206-4d7d-a347-ea22421a5af9 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:41.914982 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbfa9a-3206-4d7d-a347-ea22421a5af9 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:41.915259 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbfa9a-3206-4d7d-a347-ea22421a5af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:41.922795 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 772/1548] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:41.925166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06449b99-e254-4a6a-84cf-8a88ba0c7663 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:41.934925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-89bbfa9a-3206-4d7d-a347-ea22421a5af9 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:41.935168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06449b99-e254-4a6a-84cf-8a88ba0c7663 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:41.936329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-89bbfa9a-3206-4d7d-a347-ea22421a5af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:41.940397 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8d97d-d327-4014-942f-93c27840b00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:41.941037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8d97d-d327-4014-942f-93c27840b00d admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:41.941326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8d97d-d327-4014-942f-93c27840b00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:41.941604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8d97d-d327-4014-942f-93c27840b00d admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:41.942265 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8d97d-d327-4014-942f-93c27840b00d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:41.944182 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8d97d-d327-4014-942f-93c27840b00d admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:41.944752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 777/1549] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:41.945958 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d11e08-63b1-4550-96ea-cef43623a55e None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:41.946245 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d11e08-63b1-4550-96ea-cef43623a55e None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:41.946558 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d11e08-63b1-4550-96ea-cef43623a55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:41.965872 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-f2d11e08-63b1-4550-96ea-cef43623a55e None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:41.967111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f2d11e08-63b1-4550-96ea-cef43623a55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:41.968423 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 773/1550] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:35:41 2022] POST /identity/v3/users => generated 290 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:35:41.969810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46abd29-98d7-4d55-a2ef-290bfeef72f3 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:41.970095 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46abd29-98d7-4d55-a2ef-290bfeef72f3 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:41.970371 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46abd29-98d7-4d55-a2ef-290bfeef72f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:41.977325 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 778/1551] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:41.979850 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c23d8-175e-4284-a3dc-af1bc961a1a7 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:41.980147 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c23d8-175e-4284-a3dc-af1bc961a1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:41.980428 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c23d8-175e-4284-a3dc-af1bc961a1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:41.990371 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-f46abd29-98d7-4d55-a2ef-290bfeef72f3 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:41.991620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-f46abd29-98d7-4d55-a2ef-290bfeef72f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:42.005231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 774/1552] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:42.005231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01012f66-7fe7-481a-9727-6f553a49da1a None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.005231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01012f66-7fe7-481a-9727-6f553a49da1a None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.005231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01012f66-7fe7-481a-9727-6f553a49da1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.006041 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-187c23d8-175e-4284-a3dc-af1bc961a1a7 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:42.007301 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-187c23d8-175e-4284-a3dc-af1bc961a1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:42.016615 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 779/1553] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:42.017928 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c23d8-175e-4284-a3dc-af1bc961a1a7 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.028975 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c23d8-175e-4284-a3dc-af1bc961a1a7 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.030122 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-01012f66-7fe7-481a-9727-6f553a49da1a None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:42.031532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-01012f66-7fe7-481a-9727-6f553a49da1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:42.033820 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915397ba-76ef-456e-8688-f0b16fac1690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.034443 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915397ba-76ef-456e-8688-f0b16fac1690 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.034729 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915397ba-76ef-456e-8688-f0b16fac1690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.035005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915397ba-76ef-456e-8688-f0b16fac1690 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.035729 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915397ba-76ef-456e-8688-f0b16fac1690 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:35:42.036036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915397ba-76ef-456e-8688-f0b16fac1690 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.037146 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: WARNING py.warnings [None req-915397ba-76ef-456e-8688-f0b16fac1690 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 16:35:42.037146 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: warnings.warn(msg) Jul 27 16:35:42.037146 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: Jul 27 16:35:42.038015 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915397ba-76ef-456e-8688-f0b16fac1690 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:42.040701 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 775/1554] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:42.042069 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b542267-b68b-4b12-8c66-4b1f0453efc6 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.042462 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b542267-b68b-4b12-8c66-4b1f0453efc6 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.042876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b542267-b68b-4b12-8c66-4b1f0453efc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.050789 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 780/1555] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:35:42.051674 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915397ba-76ef-456e-8688-f0b16fac1690 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.062726 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915397ba-76ef-456e-8688-f0b16fac1690 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.064950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-1b542267-b68b-4b12-8c66-4b1f0453efc6 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:42.066178 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-1b542267-b68b-4b12-8c66-4b1f0453efc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:42.067896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e104cff-dcde-4d37-b9fa-2c699deba5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.068523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e104cff-dcde-4d37-b9fa-2c699deba5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.068809 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e104cff-dcde-4d37-b9fa-2c699deba5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.069111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e104cff-dcde-4d37-b9fa-2c699deba5f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.069800 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e104cff-dcde-4d37-b9fa-2c699deba5f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:35:42.070102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e104cff-dcde-4d37-b9fa-2c699deba5f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.071690 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e104cff-dcde-4d37-b9fa-2c699deba5f4 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:42.074987 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 776/1556] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:42.075771 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b542267-b68b-4b12-8c66-4b1f0453efc6 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.083925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 781/1557] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:35:42.084753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e104cff-dcde-4d37-b9fa-2c699deba5f4 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.086240 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b542267-b68b-4b12-8c66-4b1f0453efc6 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.091645 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374ffe9-5c48-4c61-a2ca-bce291355005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.092253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374ffe9-5c48-4c61-a2ca-bce291355005 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.092531 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374ffe9-5c48-4c61-a2ca-bce291355005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.092803 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374ffe9-5c48-4c61-a2ca-bce291355005 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.093592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374ffe9-5c48-4c61-a2ca-bce291355005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.094669 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e104cff-dcde-4d37-b9fa-2c699deba5f4 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.095300 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374ffe9-5c48-4c61-a2ca-bce291355005 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:42.099290 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630d402-e18b-49ce-828b-e22e86a54147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.099883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630d402-e18b-49ce-828b-e22e86a54147 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.100167 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630d402-e18b-49ce-828b-e22e86a54147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.100447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630d402-e18b-49ce-828b-e22e86a54147 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.101150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630d402-e18b-49ce-828b-e22e86a54147 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:35:42.101596 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630d402-e18b-49ce-828b-e22e86a54147 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.103179 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630d402-e18b-49ce-828b-e22e86a54147 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:42.105750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 777/1558] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:35:42.106763 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703646ba-03f5-4f05-8551-37670f1300e0 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.107064 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703646ba-03f5-4f05-8551-37670f1300e0 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.107350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703646ba-03f5-4f05-8551-37670f1300e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.115666 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 782/1559] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:35:42.116994 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5aa5b-9005-448d-9a29-25748a40560d None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.117344 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5aa5b-9005-448d-9a29-25748a40560d None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.117698 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5aa5b-9005-448d-9a29-25748a40560d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.133920 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-703646ba-03f5-4f05-8551-37670f1300e0 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:42.135132 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-703646ba-03f5-4f05-8551-37670f1300e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:42.137303 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-dda5aa5b-9005-448d-9a29-25748a40560d None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:42.138529 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-dda5aa5b-9005-448d-9a29-25748a40560d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:42.143007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 778/1560] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:42.144191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31cce4-b8d5-4e62-a1c6-5123a0ee9256 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.144476 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31cce4-b8d5-4e62-a1c6-5123a0ee9256 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.144750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed31cce4-b8d5-4e62-a1c6-5123a0ee9256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.152385 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 783/1561] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:42.152385 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5aa5b-9005-448d-9a29-25748a40560d None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.162345 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5aa5b-9005-448d-9a29-25748a40560d None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.167085 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abeaab3-a80c-47ba-b441-e9dd742e1d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.167782 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abeaab3-a80c-47ba-b441-e9dd742e1d3d admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.168509 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abeaab3-a80c-47ba-b441-e9dd742e1d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.168509 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abeaab3-a80c-47ba-b441-e9dd742e1d3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.168687 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-ed31cce4-b8d5-4e62-a1c6-5123a0ee9256 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:42.169088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abeaab3-a80c-47ba-b441-e9dd742e1d3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.169738 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ed31cce4-b8d5-4e62-a1c6-5123a0ee9256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:42.171991 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abeaab3-a80c-47ba-b441-e9dd742e1d3d admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:42.178276 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 779/1562] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:42.179129 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed31cce4-b8d5-4e62-a1c6-5123a0ee9256 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.189162 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed31cce4-b8d5-4e62-a1c6-5123a0ee9256 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.195584 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05ac3e-f749-4ec8-9ffa-19ddc3bfc497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.196234 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05ac3e-f749-4ec8-9ffa-19ddc3bfc497 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.196546 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05ac3e-f749-4ec8-9ffa-19ddc3bfc497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.196849 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05ac3e-f749-4ec8-9ffa-19ddc3bfc497 admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.197598 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05ac3e-f749-4ec8-9ffa-19ddc3bfc497 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:35:42.197942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05ac3e-f749-4ec8-9ffa-19ddc3bfc497 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.199597 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05ac3e-f749-4ec8-9ffa-19ddc3bfc497 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:42.201406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 784/1563] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:35:42 2022] POST /identity/v3/projects => generated 389 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:35:42.202372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abeaab3-a80c-47ba-b441-e9dd742e1d3d admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.213047 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 780/1564] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:35:42.214124 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05ac3e-f749-4ec8-9ffa-19ddc3bfc497 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.225129 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05ac3e-f749-4ec8-9ffa-19ddc3bfc497 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.235162 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:35:42.270216 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abeaab3-a80c-47ba-b441-e9dd742e1d3d admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.273257 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:35:42.275696 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3be4e1-3606-4861-bebe-04c82788f8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.276421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3be4e1-3606-4861-bebe-04c82788f8af admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.276702 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3be4e1-3606-4861-bebe-04c82788f8af admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.277126 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3be4e1-3606-4861-bebe-04c82788f8af admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.277676 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3be4e1-3606-4861-bebe-04c82788f8af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.280451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3be4e1-3606-4861-bebe-04c82788f8af admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:42.283831 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca97431-d132-4a83-891c-1aa6eaddd886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.284448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca97431-d132-4a83-891c-1aa6eaddd886 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.284730 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca97431-d132-4a83-891c-1aa6eaddd886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.285055 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca97431-d132-4a83-891c-1aa6eaddd886 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.286027 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca97431-d132-4a83-891c-1aa6eaddd886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.287433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca97431-d132-4a83-891c-1aa6eaddd886 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:42.298182 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 781/1565] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:35:42.298998 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca97431-d132-4a83-891c-1aa6eaddd886 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.309064 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca97431-d132-4a83-891c-1aa6eaddd886 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.309815 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 785/1566] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:35:42 2022] POST /identity/v3/projects => generated 391 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:35:42.310624 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3be4e1-3606-4861-bebe-04c82788f8af admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.370279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fef9e-613c-43a3-b06a-0ce035bc6a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.371002 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fef9e-613c-43a3-b06a-0ce035bc6a46 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.371290 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fef9e-613c-43a3-b06a-0ce035bc6a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.371570 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fef9e-613c-43a3-b06a-0ce035bc6a46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.372228 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5fef9e-613c-43a3-b06a-0ce035bc6a46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.372934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3be4e1-3606-4861-bebe-04c82788f8af admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.373315 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: WARNING py.warnings [None req-7d5fef9e-613c-43a3-b06a-0ce035bc6a46 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 16:35:42.373315 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: warnings.warn(msg) Jul 27 16:35:42.373315 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: Jul 27 16:35:42.374565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5fef9e-613c-43a3-b06a-0ce035bc6a46 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:42.378073 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4f5cf-0857-4e10-8329-377e3a87d921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.378763 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4f5cf-0857-4e10-8329-377e3a87d921 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.379121 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4f5cf-0857-4e10-8329-377e3a87d921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.379466 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4f5cf-0857-4e10-8329-377e3a87d921 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.380223 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4f5cf-0857-4e10-8329-377e3a87d921 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:35:42.380601 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4f5cf-0857-4e10-8329-377e3a87d921 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.382269 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4f5cf-0857-4e10-8329-377e3a87d921 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:42.393682 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 786/1567] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:35:42.394477 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4f5cf-0857-4e10-8329-377e3a87d921 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.406304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 782/1568] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:35:42 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:35:42.406501 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4f5cf-0857-4e10-8329-377e3a87d921 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.406617 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fef9e-613c-43a3-b06a-0ce035bc6a46 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.461780 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa6390-f49b-4227-865c-180fba12799f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.462673 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa6390-f49b-4227-865c-180fba12799f admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.462981 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa6390-f49b-4227-865c-180fba12799f admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.463257 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa6390-f49b-4227-865c-180fba12799f admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.463948 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa6390-f49b-4227-865c-180fba12799f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:35:42.464293 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa6390-f49b-4227-865c-180fba12799f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.466055 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa6390-f49b-4227-865c-180fba12799f admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:42.467879 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fef9e-613c-43a3-b06a-0ce035bc6a46 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.472566 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17fe23-8e0f-4f3a-a034-3d6e4ef09a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.473276 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17fe23-8e0f-4f3a-a034-3d6e4ef09a21 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.473629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17fe23-8e0f-4f3a-a034-3d6e4ef09a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.473981 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17fe23-8e0f-4f3a-a034-3d6e4ef09a21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.474811 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17fe23-8e0f-4f3a-a034-3d6e4ef09a21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:35:42.475517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17fe23-8e0f-4f3a-a034-3d6e4ef09a21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.477218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17fe23-8e0f-4f3a-a034-3d6e4ef09a21 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:42.479101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 787/1569] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:35:42.479980 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa6390-f49b-4227-865c-180fba12799f admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.490627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa6390-f49b-4227-865c-180fba12799f admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.495512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e34c81-9d53-4035-abe5-160eb291c353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.496123 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e34c81-9d53-4035-abe5-160eb291c353 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.496595 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e34c81-9d53-4035-abe5-160eb291c353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.496867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e34c81-9d53-4035-abe5-160eb291c353 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.497546 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e34c81-9d53-4035-abe5-160eb291c353 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.497704 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 783/1570] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:35:42.498507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17fe23-8e0f-4f3a-a034-3d6e4ef09a21 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.498726 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: WARNING py.warnings [None req-48e34c81-9d53-4035-abe5-160eb291c353 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 16:35:42.498726 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: warnings.warn(msg) Jul 27 16:35:42.498726 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: Jul 27 16:35:42.500162 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e34c81-9d53-4035-abe5-160eb291c353 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:42.511226 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17fe23-8e0f-4f3a-a034-3d6e4ef09a21 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.518491 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae3be1-2d94-47ad-ad4d-448fc7bad003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.519446 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae3be1-2d94-47ad-ad4d-448fc7bad003 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.519876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae3be1-2d94-47ad-ad4d-448fc7bad003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.520285 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae3be1-2d94-47ad-ad4d-448fc7bad003 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.521385 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae3be1-2d94-47ad-ad4d-448fc7bad003 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.526517 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 788/1571] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:35:42 2022] POST /identity/v3/users => generated 295 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:35:42.526517 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e34c81-9d53-4035-abe5-160eb291c353 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.526765 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae3be1-2d94-47ad-ad4d-448fc7bad003 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:42.537098 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e34c81-9d53-4035-abe5-160eb291c353 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.542882 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f0ea5-4993-4539-9aa3-1421f929d3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.543557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f0ea5-4993-4539-9aa3-1421f929d3ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.543842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f0ea5-4993-4539-9aa3-1421f929d3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.544115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f0ea5-4993-4539-9aa3-1421f929d3ed admin admin] PATH_INFO: `/v3/projects/9f922d21e1e8434ea5e93938dc56bed9/users/c5710d1f3a6742e489c7c676cc4c01d3/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.558116 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 784/1572] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:35:42 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:35:42.558980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae3be1-2d94-47ad-ad4d-448fc7bad003 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.569376 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f0ea5-4993-4539-9aa3-1421f929d3ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f922d21e1e8434ea5e93938dc56bed9, user_id=c5710d1f3a6742e489c7c676cc4c01d3, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.572636 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f0ea5-4993-4539-9aa3-1421f929d3ed admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:42.582804 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:35:42.588545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 789/1573] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:35:42 2022] PUT /identity/v3/projects/9f922d21e1e8434ea5e93938dc56bed9/users/c5710d1f3a6742e489c7c676cc4c01d3/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:35:42.589476 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f0ea5-4993-4539-9aa3-1421f929d3ed admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.614221 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:35:42.629639 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae3be1-2d94-47ad-ad4d-448fc7bad003 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.653857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f0ea5-4993-4539-9aa3-1421f929d3ed admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.659166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fead4-7cf1-4261-98ef-c56af50118b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.659780 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fead4-7cf1-4261-98ef-c56af50118b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.660070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fead4-7cf1-4261-98ef-c56af50118b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.660350 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fead4-7cf1-4261-98ef-c56af50118b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.661066 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0fead4-7cf1-4261-98ef-c56af50118b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.662989 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0fead4-7cf1-4261-98ef-c56af50118b6 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:42.687101 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de51eb-2d3b-4f66-8c41-16d617c2c908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.687728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de51eb-2d3b-4f66-8c41-16d617c2c908 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.688007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de51eb-2d3b-4f66-8c41-16d617c2c908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.688281 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de51eb-2d3b-4f66-8c41-16d617c2c908 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.688960 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de51eb-2d3b-4f66-8c41-16d617c2c908 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.689201 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 790/1574] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:35:42 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:35:42.690224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fead4-7cf1-4261-98ef-c56af50118b6 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.690925 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de51eb-2d3b-4f66-8c41-16d617c2c908 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:42.713082 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 785/1575] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:35:42 2022] POST /identity/v3/users => generated 296 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:35:42.713878 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de51eb-2d3b-4f66-8c41-16d617c2c908 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.723943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de51eb-2d3b-4f66-8c41-16d617c2c908 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.758961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fead4-7cf1-4261-98ef-c56af50118b6 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.763803 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe30af1-4bd1-4e78-921d-e0dcaa93f915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.764985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe30af1-4bd1-4e78-921d-e0dcaa93f915 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.765280 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe30af1-4bd1-4e78-921d-e0dcaa93f915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.765602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe30af1-4bd1-4e78-921d-e0dcaa93f915 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.766316 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe30af1-4bd1-4e78-921d-e0dcaa93f915 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.768257 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe30af1-4bd1-4e78-921d-e0dcaa93f915 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:42.788472 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f1cb2-c12b-4561-bf46-b6b2d1ee06cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.789320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f1cb2-c12b-4561-bf46-b6b2d1ee06cb admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.789529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f1cb2-c12b-4561-bf46-b6b2d1ee06cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.789805 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f1cb2-c12b-4561-bf46-b6b2d1ee06cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.790476 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f1cb2-c12b-4561-bf46-b6b2d1ee06cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.792656 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f1cb2-c12b-4561-bf46-b6b2d1ee06cb admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:42.794649 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 791/1576] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:35:42 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:35:42.795451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe30af1-4bd1-4e78-921d-e0dcaa93f915 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.804957 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe30af1-4bd1-4e78-921d-e0dcaa93f915 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.809556 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c63e1-282b-4cf5-a072-cfd788a97fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.810158 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c63e1-282b-4cf5-a072-cfd788a97fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.810445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c63e1-282b-4cf5-a072-cfd788a97fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.810729 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c63e1-282b-4cf5-a072-cfd788a97fd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.811448 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c63e1-282b-4cf5-a072-cfd788a97fd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.813392 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c63e1-282b-4cf5-a072-cfd788a97fd1 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:42.824108 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 786/1577] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:35:42 2022] POST /identity/v3/projects => generated 405 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:35:42.824108 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f1cb2-c12b-4561-bf46-b6b2d1ee06cb admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.843566 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 792/1578] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:35:42 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:35:42.844457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c63e1-282b-4cf5-a072-cfd788a97fd1 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.897204 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f1cb2-c12b-4561-bf46-b6b2d1ee06cb admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.904132 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:35:42.914782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:35:42.917223 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c63e1-282b-4cf5-a072-cfd788a97fd1 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.921981 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed08f8-ca23-4eff-abc4-deef2a84246c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.922597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed08f8-ca23-4eff-abc4-deef2a84246c admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.922881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed08f8-ca23-4eff-abc4-deef2a84246c admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.923156 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed08f8-ca23-4eff-abc4-deef2a84246c admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.923817 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed08f8-ca23-4eff-abc4-deef2a84246c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.925896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed08f8-ca23-4eff-abc4-deef2a84246c admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:42.949829 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 793/1579] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:35:42 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:35:42.951117 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8e76b-1d38-4728-b175-6b18ec877bf4 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.951388 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8e76b-1d38-4728-b175-6b18ec877bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.951666 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8e76b-1d38-4728-b175-6b18ec877bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.965752 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cc8e9-8211-4902-87ec-e5ae16a63416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.966403 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cc8e9-8211-4902-87ec-e5ae16a63416 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.966689 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cc8e9-8211-4902-87ec-e5ae16a63416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.966965 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cc8e9-8211-4902-87ec-e5ae16a63416 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.968286 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76cc8e9-8211-4902-87ec-e5ae16a63416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.969969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76cc8e9-8211-4902-87ec-e5ae16a63416 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:42.971408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-6ce8e76b-1d38-4728-b175-6b18ec877bf4 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:42.972638 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-6ce8e76b-1d38-4728-b175-6b18ec877bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:42.982704 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 787/1580] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:35:42.982704 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cc8e9-8211-4902-87ec-e5ae16a63416 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.982950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 794/1581] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:42.982950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8e76b-1d38-4728-b175-6b18ec877bf4 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:42.992032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8e76b-1d38-4728-b175-6b18ec877bf4 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.994321 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cc8e9-8211-4902-87ec-e5ae16a63416 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:42.996752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb3c4b-b6de-4752-ab1b-0707b78fcfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.997431 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb3c4b-b6de-4752-ab1b-0707b78fcfae admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.997713 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb3c4b-b6de-4752-ab1b-0707b78fcfae admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:42.997988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb3c4b-b6de-4752-ab1b-0707b78fcfae admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:42.998738 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb3c4b-b6de-4752-ab1b-0707b78fcfae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:42.999008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c882ac-7a52-41cd-a37a-c4d35db58f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:42.999613 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c882ac-7a52-41cd-a37a-c4d35db58f2f admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:42.999893 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c882ac-7a52-41cd-a37a-c4d35db58f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.000165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c882ac-7a52-41cd-a37a-c4d35db58f2f admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.000369 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb3c4b-b6de-4752-ab1b-0707b78fcfae admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.000816 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c882ac-7a52-41cd-a37a-c4d35db58f2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.002742 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c882ac-7a52-41cd-a37a-c4d35db58f2f admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.010391 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 795/1582] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:35:43.011378 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb3c4b-b6de-4752-ab1b-0707b78fcfae admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.024469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb3c4b-b6de-4752-ab1b-0707b78fcfae admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.028453 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 788/1583] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:35:42 2022] POST /identity/v3/users => generated 288 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:35:43.028453 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c882ac-7a52-41cd-a37a-c4d35db58f2f admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.029247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226ecf5-2490-46da-a991-82ee7e036ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.030007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226ecf5-2490-46da-a991-82ee7e036ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.030928 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226ecf5-2490-46da-a991-82ee7e036ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.031210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226ecf5-2490-46da-a991-82ee7e036ec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.031901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226ecf5-2490-46da-a991-82ee7e036ec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.033538 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226ecf5-2490-46da-a991-82ee7e036ec6 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.036244 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c882ac-7a52-41cd-a37a-c4d35db58f2f admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.041366 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6567a-4f06-48cf-a33f-716895876981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.041905 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6567a-4f06-48cf-a33f-716895876981 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.042190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6567a-4f06-48cf-a33f-716895876981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.042487 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6567a-4f06-48cf-a33f-716895876981 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.043132 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6567a-4f06-48cf-a33f-716895876981 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.046505 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 796/1584] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:35:43.046505 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226ecf5-2490-46da-a991-82ee7e036ec6 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.046739 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6567a-4f06-48cf-a33f-716895876981 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.054010 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226ecf5-2490-46da-a991-82ee7e036ec6 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.059012 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187db427-36c7-44c3-83a4-68ffa0df486c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.059730 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187db427-36c7-44c3-83a4-68ffa0df486c admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.060018 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187db427-36c7-44c3-83a4-68ffa0df486c admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.060295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187db427-36c7-44c3-83a4-68ffa0df486c admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.060968 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187db427-36c7-44c3-83a4-68ffa0df486c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.063419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187db427-36c7-44c3-83a4-68ffa0df486c admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.068610 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 789/1585] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:35:43 2022] POST /identity/v3/users => generated 303 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:35:43.072478 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6567a-4f06-48cf-a33f-716895876981 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.083005 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6567a-4f06-48cf-a33f-716895876981 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.087381 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 797/1586] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:35:43 2022] POST /identity/v3/users => generated 293 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:35:43.087381 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187db427-36c7-44c3-83a4-68ffa0df486c admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.088119 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ea1c6-5c5c-4927-9cee-f60460f5e237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.088794 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ea1c6-5c5c-4927-9cee-f60460f5e237 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.089101 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ea1c6-5c5c-4927-9cee-f60460f5e237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.089376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ea1c6-5c5c-4927-9cee-f60460f5e237 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.090095 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ea1c6-5c5c-4927-9cee-f60460f5e237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.091681 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ea1c6-5c5c-4927-9cee-f60460f5e237 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.095376 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187db427-36c7-44c3-83a4-68ffa0df486c admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.100002 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767af231-1832-4228-a131-859845ae6954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.100598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767af231-1832-4228-a131-859845ae6954 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.100882 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767af231-1832-4228-a131-859845ae6954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.101190 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767af231-1832-4228-a131-859845ae6954 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.101878 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767af231-1832-4228-a131-859845ae6954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.102298 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 790/1587] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:35:43.103148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ea1c6-5c5c-4927-9cee-f60460f5e237 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.103482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767af231-1832-4228-a131-859845ae6954 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.113728 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 798/1588] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:35:43.114517 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767af231-1832-4228-a131-859845ae6954 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.126459 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767af231-1832-4228-a131-859845ae6954 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.131319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b4392-7f06-47aa-9a9a-65b510f8ae20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.132057 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b4392-7f06-47aa-9a9a-65b510f8ae20 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.132405 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b4392-7f06-47aa-9a9a-65b510f8ae20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.132792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b4392-7f06-47aa-9a9a-65b510f8ae20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.133567 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b4392-7f06-47aa-9a9a-65b510f8ae20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.135377 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b4392-7f06-47aa-9a9a-65b510f8ae20 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.145561 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 799/1589] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:35:43.149519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b4392-7f06-47aa-9a9a-65b510f8ae20 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.158348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ea1c6-5c5c-4927-9cee-f60460f5e237 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.159579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b4392-7f06-47aa-9a9a-65b510f8ae20 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.162186 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42379a60-ee2e-45d3-903a-d76247550a5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.166270 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42379a60-ee2e-45d3-903a-d76247550a5e service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.166270 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42379a60-ee2e-45d3-903a-d76247550a5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.166270 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42379a60-ee2e-45d3-903a-d76247550a5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.166568 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24a332-7fc2-4f93-8f2d-230ab4353090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.166568 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24a332-7fc2-4f93-8f2d-230ab4353090 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.166568 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24a332-7fc2-4f93-8f2d-230ab4353090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.166568 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24a332-7fc2-4f93-8f2d-230ab4353090 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.166899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb24a332-7fc2-4f93-8f2d-230ab4353090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.167624 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb24a332-7fc2-4f93-8f2d-230ab4353090 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.173477 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42379a60-ee2e-45d3-903a-d76247550a5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.174951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42379a60-ee2e-45d3-903a-d76247550a5e service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.179005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 800/1590] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:35:43.179806 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24a332-7fc2-4f93-8f2d-230ab4353090 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.187159 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 791/1591] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:35:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:35:43.187969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42379a60-ee2e-45d3-903a-d76247550a5e service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.194706 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24a332-7fc2-4f93-8f2d-230ab4353090 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.197429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42379a60-ee2e-45d3-903a-d76247550a5e service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.199554 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad03c2-6ab4-4fa8-9599-481ca7fb80d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.200250 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad03c2-6ab4-4fa8-9599-481ca7fb80d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.200614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad03c2-6ab4-4fa8-9599-481ca7fb80d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.201484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad03c2-6ab4-4fa8-9599-481ca7fb80d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.201802 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad03c2-6ab4-4fa8-9599-481ca7fb80d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.203523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad03c2-6ab4-4fa8-9599-481ca7fb80d7 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.205268 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8dbc2-64be-439e-a6aa-b127f8f76013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.205896 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8dbc2-64be-439e-a6aa-b127f8f76013 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.206338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8dbc2-64be-439e-a6aa-b127f8f76013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.206463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8dbc2-64be-439e-a6aa-b127f8f76013 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.207139 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8dbc2-64be-439e-a6aa-b127f8f76013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.208756 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8dbc2-64be-439e-a6aa-b127f8f76013 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.214970 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 801/1592] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:35:43.219112 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad03c2-6ab4-4fa8-9599-481ca7fb80d7 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.224940 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 792/1593] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:35:43.226947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8dbc2-64be-439e-a6aa-b127f8f76013 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.235781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad03c2-6ab4-4fa8-9599-481ca7fb80d7 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.240336 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8dbc2-64be-439e-a6aa-b127f8f76013 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.241564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e11c4-2f0b-43e2-8a41-bbcf7fb61888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.242222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e11c4-2f0b-43e2-8a41-bbcf7fb61888 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.242676 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e11c4-2f0b-43e2-8a41-bbcf7fb61888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.242954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e11c4-2f0b-43e2-8a41-bbcf7fb61888 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.243674 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e11c4-2f0b-43e2-8a41-bbcf7fb61888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.245482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46171b-3870-408f-90c8-6e5ab6158c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.245612 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e11c4-2f0b-43e2-8a41-bbcf7fb61888 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.246105 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46171b-3870-408f-90c8-6e5ab6158c87 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.246390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46171b-3870-408f-90c8-6e5ab6158c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.246667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46171b-3870-408f-90c8-6e5ab6158c87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.247365 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc46171b-3870-408f-90c8-6e5ab6158c87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.249059 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc46171b-3870-408f-90c8-6e5ab6158c87 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.259623 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 802/1594] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:35:43.259623 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e11c4-2f0b-43e2-8a41-bbcf7fb61888 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.267095 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 793/1595] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:35:43.267095 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46171b-3870-408f-90c8-6e5ab6158c87 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.272260 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e11c4-2f0b-43e2-8a41-bbcf7fb61888 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.277721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46171b-3870-408f-90c8-6e5ab6158c87 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.278239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4200c3d-d34a-424c-a241-8b5f69d057ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.279177 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4200c3d-d34a-424c-a241-8b5f69d057ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.279627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4200c3d-d34a-424c-a241-8b5f69d057ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.280762 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4200c3d-d34a-424c-a241-8b5f69d057ea admin admin] PATH_INFO: `/v3/projects/8d570044769a44d9b18c905284cd9947/users/22bb7775bae64374b11462a53853e54f/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.285872 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b07b1f-a430-4916-8044-f1f7a69ce108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.286960 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b07b1f-a430-4916-8044-f1f7a69ce108 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.287418 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b07b1f-a430-4916-8044-f1f7a69ce108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.287893 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b07b1f-a430-4916-8044-f1f7a69ce108 admin admin] PATH_INFO: `/v3/projects/027a003da80c43ca9b4b337527b93d3a/users/1c0838069cba4d1389ec6fc809db70a7/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.301353 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4200c3d-d34a-424c-a241-8b5f69d057ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d570044769a44d9b18c905284cd9947, user_id=22bb7775bae64374b11462a53853e54f, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.304277 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4200c3d-d34a-424c-a241-8b5f69d057ea admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.306879 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b07b1f-a430-4916-8044-f1f7a69ce108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=027a003da80c43ca9b4b337527b93d3a, user_id=1c0838069cba4d1389ec6fc809db70a7, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.307984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: WARNING py.warnings [None req-23b07b1f-a430-4916-8044-f1f7a69ce108 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 16:35:43.307984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: warnings.warn(msg) Jul 27 16:35:43.307984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: Jul 27 16:35:43.310122 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b07b1f-a430-4916-8044-f1f7a69ce108 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.319071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 803/1596] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:35:43 2022] PUT /identity/v3/projects/8d570044769a44d9b18c905284cd9947/users/22bb7775bae64374b11462a53853e54f/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:35:43.320371 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4200c3d-d34a-424c-a241-8b5f69d057ea admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.327169 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 794/1597] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:35:43 2022] PUT /identity/v3/projects/027a003da80c43ca9b4b337527b93d3a/users/1c0838069cba4d1389ec6fc809db70a7/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:35:43.328315 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b07b1f-a430-4916-8044-f1f7a69ce108 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.361658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:35:43.394341 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4200c3d-d34a-424c-a241-8b5f69d057ea admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.406995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b07b1f-a430-4916-8044-f1f7a69ce108 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.411786 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccc3c8f-627e-4662-bdaa-4f7f030dca79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.412755 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc3c8f-627e-4662-bdaa-4f7f030dca79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.413427 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc3c8f-627e-4662-bdaa-4f7f030dca79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.413831 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc3c8f-627e-4662-bdaa-4f7f030dca79 admin admin] PATH_INFO: `/v3/projects/27f1eace103144f197997ef13c78c7ca/users/3ada16d9d23b4e80a99df9ffc53f2185/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.416555 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:35:43.432435 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccc3c8f-627e-4662-bdaa-4f7f030dca79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27f1eace103144f197997ef13c78c7ca, user_id=3ada16d9d23b4e80a99df9ffc53f2185, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.435326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccc3c8f-627e-4662-bdaa-4f7f030dca79 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.451324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 795/1598] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:35:43 2022] PUT /identity/v3/projects/27f1eace103144f197997ef13c78c7ca/users/3ada16d9d23b4e80a99df9ffc53f2185/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:35:43.462557 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccc3c8f-627e-4662-bdaa-4f7f030dca79 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.472275 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611e1ff-d61e-4c89-ac3f-a9bc067f308b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.473314 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611e1ff-d61e-4c89-ac3f-a9bc067f308b admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.473703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611e1ff-d61e-4c89-ac3f-a9bc067f308b admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.474133 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611e1ff-d61e-4c89-ac3f-a9bc067f308b admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.475194 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1611e1ff-d61e-4c89-ac3f-a9bc067f308b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.477217 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1611e1ff-d61e-4c89-ac3f-a9bc067f308b admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.493114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 804/1599] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:35:43.493114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611e1ff-d61e-4c89-ac3f-a9bc067f308b admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.503348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611e1ff-d61e-4c89-ac3f-a9bc067f308b admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.533227 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccc3c8f-627e-4662-bdaa-4f7f030dca79 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.538069 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911526b-1328-43a6-96cf-50d49f327f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.538694 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911526b-1328-43a6-96cf-50d49f327f45 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.538987 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911526b-1328-43a6-96cf-50d49f327f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.539263 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911526b-1328-43a6-96cf-50d49f327f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.540055 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911526b-1328-43a6-96cf-50d49f327f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.541651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911526b-1328-43a6-96cf-50d49f327f45 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.553358 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 796/1600] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:35:43.554196 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911526b-1328-43a6-96cf-50d49f327f45 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.568160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911526b-1328-43a6-96cf-50d49f327f45 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.572934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daef7d6-ad45-4c23-aad3-757d36a7b157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.573622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daef7d6-ad45-4c23-aad3-757d36a7b157 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.573903 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daef7d6-ad45-4c23-aad3-757d36a7b157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.574183 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daef7d6-ad45-4c23-aad3-757d36a7b157 admin admin] PATH_INFO: `/v3/projects/7a3e43f9afcd49ce89b5911eef170bce/users/2cc374498a024bd184d024308cc313e8/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.588510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0bdd21-8126-4171-8fec-b6dca2d1723c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.589571 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bdd21-8126-4171-8fec-b6dca2d1723c admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.589912 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bdd21-8126-4171-8fec-b6dca2d1723c admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.590377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daef7d6-ad45-4c23-aad3-757d36a7b157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a3e43f9afcd49ce89b5911eef170bce, user_id=2cc374498a024bd184d024308cc313e8, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.590563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bdd21-8126-4171-8fec-b6dca2d1723c admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.591338 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0bdd21-8126-4171-8fec-b6dca2d1723c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.593121 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daef7d6-ad45-4c23-aad3-757d36a7b157 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.593282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0bdd21-8126-4171-8fec-b6dca2d1723c admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.609322 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 805/1601] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:35:43.609322 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872a595-d315-4cd7-8cba-622cf27998cb None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.609322 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872a595-d315-4cd7-8cba-622cf27998cb None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.609322 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872a595-d315-4cd7-8cba-622cf27998cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.624980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 797/1602] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:35:43 2022] PUT /identity/v3/projects/7a3e43f9afcd49ce89b5911eef170bce/users/2cc374498a024bd184d024308cc313e8/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:35:43.624980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a581a-d3f8-4ee6-afcd-34f589885d6c None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.624980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a581a-d3f8-4ee6-afcd-34f589885d6c None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.624980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a581a-d3f8-4ee6-afcd-34f589885d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.631954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-3872a595-d315-4cd7-8cba-622cf27998cb None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:43.633429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-3872a595-d315-4cd7-8cba-622cf27998cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:43.644076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-989a581a-d3f8-4ee6-afcd-34f589885d6c None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:43.645323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-989a581a-d3f8-4ee6-afcd-34f589885d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:43.698699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 806/1603] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:43.700680 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3fe3f-bad5-4ace-ae7f-5f953a934d36 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.701188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3fe3f-bad5-4ace-ae7f-5f953a934d36 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.701480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3fe3f-bad5-4ace-ae7f-5f953a934d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.716673 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 798/1604] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:43.717665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a581a-d3f8-4ee6-afcd-34f589885d6c None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.727312 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a581a-d3f8-4ee6-afcd-34f589885d6c None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.734292 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83e54f-abdd-47c8-8981-efe346481088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.734968 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83e54f-abdd-47c8-8981-efe346481088 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.735248 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83e54f-abdd-47c8-8981-efe346481088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.735522 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83e54f-abdd-47c8-8981-efe346481088 admin admin] PATH_INFO: `/v3/projects/89ddf8711f2f4be88304f967149c3a50/users/7a0bd78165c94d9e9750fe4af4f7d375/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.744691 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-abe3fe3f-bad5-4ace-ae7f-5f953a934d36 None None] MFA Rules not processed for user `3ada16d9d23b4e80a99df9ffc53f2185`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:43.746646 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-abe3fe3f-bad5-4ace-ae7f-5f953a934d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:43.751688 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83e54f-abdd-47c8-8981-efe346481088 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ddf8711f2f4be88304f967149c3a50, user_id=7a0bd78165c94d9e9750fe4af4f7d375, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.754477 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83e54f-abdd-47c8-8981-efe346481088 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.757254 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:35:43.774339 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 799/1605] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:35:43 2022] PUT /identity/v3/projects/89ddf8711f2f4be88304f967149c3a50/users/7a0bd78165c94d9e9750fe4af4f7d375/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:35:43.778818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83e54f-abdd-47c8-8981-efe346481088 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.803742 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:35:43.829539 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 807/1606] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:43.830462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3fe3f-bad5-4ace-ae7f-5f953a934d36 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.844057 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3fe3f-bad5-4ace-ae7f-5f953a934d36 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.849596 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d3144d-c803-4050-a31c-503c6e547503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.850441 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3144d-c803-4050-a31c-503c6e547503 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.850914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3144d-c803-4050-a31c-503c6e547503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.851204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3144d-c803-4050-a31c-503c6e547503 admin admin] PATH_INFO: `/v3/projects/ebcc91e5911a4a418118b3775cfe4b76/users/140059c5b30c426ab6d7f1236eb5f684/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.852979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83e54f-abdd-47c8-8981-efe346481088 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.858912 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1c63e-b0e4-4aba-b93f-3be6e205e924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.859713 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1c63e-b0e4-4aba-b93f-3be6e205e924 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.860045 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1c63e-b0e4-4aba-b93f-3be6e205e924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.860320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1c63e-b0e4-4aba-b93f-3be6e205e924 admin admin] PATH_INFO: `/v3/projects/2e682c25fdf54f80bbb9fbce41d92b77/users/92130afa772e4d6e9b3c018eb2687712/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.867367 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d3144d-c803-4050-a31c-503c6e547503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebcc91e5911a4a418118b3775cfe4b76, user_id=140059c5b30c426ab6d7f1236eb5f684, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.870854 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d3144d-c803-4050-a31c-503c6e547503 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.879508 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1c63e-b0e4-4aba-b93f-3be6e205e924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e682c25fdf54f80bbb9fbce41d92b77, user_id=92130afa772e4d6e9b3c018eb2687712, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:43.882455 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1c63e-b0e4-4aba-b93f-3be6e205e924 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:43.886385 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 808/1607] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:35:43 2022] PUT /identity/v3/projects/ebcc91e5911a4a418118b3775cfe4b76/users/140059c5b30c426ab6d7f1236eb5f684/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:35:43.887849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80674cf-7208-46db-b6cd-b45ae0c0da16 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.888140 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80674cf-7208-46db-b6cd-b45ae0c0da16 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.888418 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80674cf-7208-46db-b6cd-b45ae0c0da16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:43.897282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 800/1608] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:35:43 2022] PUT /identity/v3/projects/2e682c25fdf54f80bbb9fbce41d92b77/users/92130afa772e4d6e9b3c018eb2687712/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:35:43.898360 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1c63e-b0e4-4aba-b93f-3be6e205e924 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:43.922949 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-e80674cf-7208-46db-b6cd-b45ae0c0da16 None None] MFA Rules not processed for user `2cc374498a024bd184d024308cc313e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:43.924586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e80674cf-7208-46db-b6cd-b45ae0c0da16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:43.970927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1c63e-b0e4-4aba-b93f-3be6e205e924 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:43.975683 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd25288-b649-476a-9b2f-b13778cec691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:43.976432 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd25288-b649-476a-9b2f-b13778cec691 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:43.976828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd25288-b649-476a-9b2f-b13778cec691 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:43.977207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd25288-b649-476a-9b2f-b13778cec691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:44.002482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 809/1609] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:43 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:44.003344 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80674cf-7208-46db-b6cd-b45ae0c0da16 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:44.018313 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80674cf-7208-46db-b6cd-b45ae0c0da16 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:44.022331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfcdd76-61b2-4b40-aa8b-0ebf8cd12b2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:44.022981 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcdd76-61b2-4b40-aa8b-0ebf8cd12b2a service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:44.023289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcdd76-61b2-4b40-aa8b-0ebf8cd12b2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:44.023588 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcdd76-61b2-4b40-aa8b-0ebf8cd12b2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:44.054427 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd25288-b649-476a-9b2f-b13778cec691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:44.055901 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd25288-b649-476a-9b2f-b13778cec691 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:44.068438 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 801/1610] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:35:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:35:44.071966 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b54786-e86d-4f09-91c0-f43231e21fce None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:44.072293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b54786-e86d-4f09-91c0-f43231e21fce None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:44.072518 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b54786-e86d-4f09-91c0-f43231e21fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:44.116809 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfcdd76-61b2-4b40-aa8b-0ebf8cd12b2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:44.118857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfcdd76-61b2-4b40-aa8b-0ebf8cd12b2a service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:44.125017 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-d1b54786-e86d-4f09-91c0-f43231e21fce None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:44.128095 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d1b54786-e86d-4f09-91c0-f43231e21fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:44.134274 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 810/1611] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:35:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:35:44.135796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfcdd76-61b2-4b40-aa8b-0ebf8cd12b2a service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:44.146322 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 802/1612] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:44.147730 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a8fb6-c8d4-48ad-82c2-e3e7c1234b2b None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:44.148022 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a8fb6-c8d4-48ad-82c2-e3e7c1234b2b None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:44.149370 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a8fb6-c8d4-48ad-82c2-e3e7c1234b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:44.155485 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfcdd76-61b2-4b40-aa8b-0ebf8cd12b2a service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:44.169843 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15250497-e2f6-4163-9599-1b2d929c3ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:44.172113 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15250497-e2f6-4163-9599-1b2d929c3ee0 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:44.173003 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15250497-e2f6-4163-9599-1b2d929c3ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:44.175637 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15250497-e2f6-4163-9599-1b2d929c3ee0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:44.176955 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15250497-e2f6-4163-9599-1b2d929c3ee0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:44.186583 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15250497-e2f6-4163-9599-1b2d929c3ee0 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:44.203282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-df9a8fb6-c8d4-48ad-82c2-e3e7c1234b2b None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:44.208240 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-df9a8fb6-c8d4-48ad-82c2-e3e7c1234b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:44.228409 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 803/1613] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:44.236991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d657e8-bec5-4196-bd72-2934bae00b9b None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:44.237157 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d657e8-bec5-4196-bd72-2934bae00b9b None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:44.237329 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d657e8-bec5-4196-bd72-2934bae00b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:44.262211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 811/1614] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:35:44 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:35:44.263238 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15250497-e2f6-4163-9599-1b2d929c3ee0 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:44.290731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-c0d657e8-bec5-4196-bd72-2934bae00b9b None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:44.296375 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c0d657e8-bec5-4196-bd72-2934bae00b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:44.311128 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:35:44.323392 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:35:44.366880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15250497-e2f6-4163-9599-1b2d929c3ee0 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:44.371360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e2432-b244-4ff8-82b4-2359c731cb49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:44.372188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e2432-b244-4ff8-82b4-2359c731cb49 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:44.372778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e2432-b244-4ff8-82b4-2359c731cb49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:44.373257 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e2432-b244-4ff8-82b4-2359c731cb49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:44.417756 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 804/1615] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:44.424208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d657e8-bec5-4196-bd72-2934bae00b9b None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:44.476290 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e2432-b244-4ff8-82b4-2359c731cb49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:44.478215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e2432-b244-4ff8-82b4-2359c731cb49 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:44.491469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 812/1616] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:35:44 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:35:44.492136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e2432-b244-4ff8-82b4-2359c731cb49 service cinder] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:44.531410 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d657e8-bec5-4196-bd72-2934bae00b9b None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:44.535757 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7cb2c-e07c-4a61-8399-0df8634980c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:44.548939 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7cb2c-e07c-4a61-8399-0df8634980c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:44.559767 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7cb2c-e07c-4a61-8399-0df8634980c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:44.561063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7cb2c-e07c-4a61-8399-0df8634980c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:44.588442 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7cb2c-e07c-4a61-8399-0df8634980c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:44.593126 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7cb2c-e07c-4a61-8399-0df8634980c6 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:44.606352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e2432-b244-4ff8-82b4-2359c731cb49 service cinder] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:44.613747 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 805/1617] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:35:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:35:44.616111 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7cb2c-e07c-4a61-8399-0df8634980c6 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:44.625051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbc4db-ad3f-459c-bd92-641deee9ba19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:44.626020 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbc4db-ad3f-459c-bd92-641deee9ba19 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:44.627213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbc4db-ad3f-459c-bd92-641deee9ba19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:44.627696 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbc4db-ad3f-459c-bd92-641deee9ba19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:44.632398 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7cb2c-e07c-4a61-8399-0df8634980c6 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:44.638650 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230d14a-21a1-4e09-b5e6-e421058184b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:44.639415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230d14a-21a1-4e09-b5e6-e421058184b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:44.639798 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230d14a-21a1-4e09-b5e6-e421058184b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:44.640790 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230d14a-21a1-4e09-b5e6-e421058184b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:44.641667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230d14a-21a1-4e09-b5e6-e421058184b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:44.645812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230d14a-21a1-4e09-b5e6-e421058184b2 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:44.658468 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbbc4db-ad3f-459c-bd92-641deee9ba19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:44.660122 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbbc4db-ad3f-459c-bd92-641deee9ba19 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:44.686663 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 806/1618] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:35:44 2022] POST /identity/v3/users => generated 292 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:35:44.686663 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230d14a-21a1-4e09-b5e6-e421058184b2 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:44.690479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 813/1619] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:35:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:35:44.691518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbc4db-ad3f-459c-bd92-641deee9ba19 service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:44.708067 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230d14a-21a1-4e09-b5e6-e421058184b2 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:44.717196 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbc4db-ad3f-459c-bd92-641deee9ba19 service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:44.718039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b213c75-e5ab-451b-b171-58578b29c705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:44.718773 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b213c75-e5ab-451b-b171-58578b29c705 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:44.719060 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b213c75-e5ab-451b-b171-58578b29c705 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:44.719725 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b213c75-e5ab-451b-b171-58578b29c705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:44.725216 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd6f2e-dda2-4ead-9bad-7fc4c083cd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:44.726166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd6f2e-dda2-4ead-9bad-7fc4c083cd9c admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:44.726618 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd6f2e-dda2-4ead-9bad-7fc4c083cd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:44.727089 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd6f2e-dda2-4ead-9bad-7fc4c083cd9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:44.727968 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd6f2e-dda2-4ead-9bad-7fc4c083cd9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:44.729987 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd6f2e-dda2-4ead-9bad-7fc4c083cd9c admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:44.735220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b213c75-e5ab-451b-b171-58578b29c705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:44.737038 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b213c75-e5ab-451b-b171-58578b29c705 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:44.744870 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 814/1620] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:35:44.748711 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd6f2e-dda2-4ead-9bad-7fc4c083cd9c admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:44.749508 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 807/1621] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:35:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:35:44.761756 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd6f2e-dda2-4ead-9bad-7fc4c083cd9c admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:44.768647 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13afc44-ec2f-4c32-8bc6-7c912f04fa26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:44.770134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13afc44-ec2f-4c32-8bc6-7c912f04fa26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:44.770882 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13afc44-ec2f-4c32-8bc6-7c912f04fa26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:44.771488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13afc44-ec2f-4c32-8bc6-7c912f04fa26 admin admin] PATH_INFO: `/v3/projects/c6298cb7b10d45b3bf2bce21d3847a30/users/36d5144a6f254f2d99302e5b46115ad9/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:44.792612 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13afc44-ec2f-4c32-8bc6-7c912f04fa26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6298cb7b10d45b3bf2bce21d3847a30, user_id=36d5144a6f254f2d99302e5b46115ad9, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:44.797436 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13afc44-ec2f-4c32-8bc6-7c912f04fa26 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:44.817061 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 815/1622] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:35:44 2022] PUT /identity/v3/projects/c6298cb7b10d45b3bf2bce21d3847a30/users/36d5144a6f254f2d99302e5b46115ad9/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:35:44.820607 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b213c75-e5ab-451b-b171-58578b29c705 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:44.855348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b213c75-e5ab-451b-b171-58578b29c705 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:44.989989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b51a5d6-b0da-4c51-8f9f-93e17b2276e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:44.991061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51a5d6-b0da-4c51-8f9f-93e17b2276e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:44.991342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51a5d6-b0da-4c51-8f9f-93e17b2276e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:44.991692 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51a5d6-b0da-4c51-8f9f-93e17b2276e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:44.993150 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b51a5d6-b0da-4c51-8f9f-93e17b2276e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:44.995554 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b51a5d6-b0da-4c51-8f9f-93e17b2276e2 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:45.018458 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 808/1623] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:35:45.019726 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13afc44-ec2f-4c32-8bc6-7c912f04fa26 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:45.035864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13afc44-ec2f-4c32-8bc6-7c912f04fa26 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:45.048625 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d03d6-0e08-44ff-b3bd-7601bd95f320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:45.049394 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d03d6-0e08-44ff-b3bd-7601bd95f320 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:45.049777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d03d6-0e08-44ff-b3bd-7601bd95f320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:45.049970 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d03d6-0e08-44ff-b3bd-7601bd95f320 admin admin] PATH_INFO: `/v3/projects/c6298cb7b10d45b3bf2bce21d3847a30/users/36d5144a6f254f2d99302e5b46115ad9/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:45.056287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d03d6-0e08-44ff-b3bd-7601bd95f320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6298cb7b10d45b3bf2bce21d3847a30, user_id=36d5144a6f254f2d99302e5b46115ad9, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:45.064157 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d03d6-0e08-44ff-b3bd-7601bd95f320 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:45.083027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 816/1624] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:35:45 2022] PUT /identity/v3/projects/c6298cb7b10d45b3bf2bce21d3847a30/users/36d5144a6f254f2d99302e5b46115ad9/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:35:45.095567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadefe0-c1a7-41ac-a4ed-9ad6279f3bf7 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:45.096202 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadefe0-c1a7-41ac-a4ed-9ad6279f3bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:45.096694 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadefe0-c1a7-41ac-a4ed-9ad6279f3bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:45.131449 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-afadefe0-c1a7-41ac-a4ed-9ad6279f3bf7 None None] MFA Rules not processed for user `36d5144a6f254f2d99302e5b46115ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:45.132264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-afadefe0-c1a7-41ac-a4ed-9ad6279f3bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:45.142817 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:35:45.232146 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 809/1625] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:45 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:45.248312 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d03d6-0e08-44ff-b3bd-7601bd95f320 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:45.306133 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:35:45.355875 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d03d6-0e08-44ff-b3bd-7601bd95f320 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:45.364493 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102bd6e-f730-4ed0-8c23-ed9b90acf885 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:45.365434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102bd6e-f730-4ed0-8c23-ed9b90acf885 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:45.365633 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102bd6e-f730-4ed0-8c23-ed9b90acf885 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:45.366084 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102bd6e-f730-4ed0-8c23-ed9b90acf885 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:45.385274 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102bd6e-f730-4ed0-8c23-ed9b90acf885 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:45.387433 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102bd6e-f730-4ed0-8c23-ed9b90acf885 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:45.429190 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 817/1626] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:35:45 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:35:45.660454 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadefe0-c1a7-41ac-a4ed-9ad6279f3bf7 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:45.677958 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadefe0-c1a7-41ac-a4ed-9ad6279f3bf7 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:45.686242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfabf6ad-1c4b-4772-801d-de63be160642 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:45.686938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabf6ad-1c4b-4772-801d-de63be160642 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:45.687227 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabf6ad-1c4b-4772-801d-de63be160642 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:45.687506 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabf6ad-1c4b-4772-801d-de63be160642 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:45.813245 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfabf6ad-1c4b-4772-801d-de63be160642 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:45.815180 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfabf6ad-1c4b-4772-801d-de63be160642 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:45.830143 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 810/1627] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:35:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:35:46.947076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f8b03-ecf1-47cd-b441-5ed88f2f3a79 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:46.947792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f8b03-ecf1-47cd-b441-5ed88f2f3a79 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:46.948630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f8b03-ecf1-47cd-b441-5ed88f2f3a79 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:46.952269 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 818/1628] 10.222.0.74 () {54 vars in 962 bytes} [Wed Jul 27 16:35:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:35:46.970286 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea850f4b-0c00-42eb-9839-b55955b478c8 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:46.970906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea850f4b-0c00-42eb-9839-b55955b478c8 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:46.971504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea850f4b-0c00-42eb-9839-b55955b478c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:47.078057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-ea850f4b-0c00-42eb-9839-b55955b478c8 None None] MFA Rules not processed for user `56dfafd15e284b7986ba07dd3819bf65`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:47.079776 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ea850f4b-0c00-42eb-9839-b55955b478c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:47.277212 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 811/1629] 10.222.0.74 () {58 vars in 1059 bytes} [Wed Jul 27 16:35:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:47.285872 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85285f2d-f447-4196-8e77-21cba718dee0 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:47.286255 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85285f2d-f447-4196-8e77-21cba718dee0 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:47.286630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85285f2d-f447-4196-8e77-21cba718dee0 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:47.288124 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 819/1630] 10.222.0.74 () {54 vars in 912 bytes} [Wed Jul 27 16:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:35:47.304421 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bcbba-3185-4336-90a8-806c1e295c15 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:47.304709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bcbba-3185-4336-90a8-806c1e295c15 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:47.305016 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bcbba-3185-4336-90a8-806c1e295c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:47.334084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-297bcbba-3185-4336-90a8-806c1e295c15 None None] MFA Rules not processed for user `56dfafd15e284b7986ba07dd3819bf65`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:47.335511 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-297bcbba-3185-4336-90a8-806c1e295c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:47.349299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 812/1631] 10.222.0.74 () {58 vars in 1009 bytes} [Wed Jul 27 16:35:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:47.365238 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85285f2d-f447-4196-8e77-21cba718dee0 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:47.383759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85285f2d-f447-4196-8e77-21cba718dee0 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:47.391528 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b4ba9-0093-4574-af07-a1e281521e05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:47.392266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b4ba9-0093-4574-af07-a1e281521e05 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:47.392548 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b4ba9-0093-4574-af07-a1e281521e05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:47.392820 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b4ba9-0093-4574-af07-a1e281521e05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:47.417943 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b4ba9-0093-4574-af07-a1e281521e05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:47.419519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b4ba9-0093-4574-af07-a1e281521e05 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:47.449203 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 820/1632] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:35:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:35:48.794238 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58d81b-e51c-4756-9beb-12cff27f2e21 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:48.794841 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58d81b-e51c-4756-9beb-12cff27f2e21 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:48.794841 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58d81b-e51c-4756-9beb-12cff27f2e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:48.866560 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-5b58d81b-e51c-4756-9beb-12cff27f2e21 None None] MFA Rules not processed for user `92130afa772e4d6e9b3c018eb2687712`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:48.869505 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-5b58d81b-e51c-4756-9beb-12cff27f2e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:48.878837 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:35:48.999061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 813/1633] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:48 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:49.000568 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b4ba9-0093-4574-af07-a1e281521e05 service cinder] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:49.246219 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b4ba9-0093-4574-af07-a1e281521e05 service cinder] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:49.257393 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa0d36-0eb3-4b80-93de-0970fa2786ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:49.258097 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa0d36-0eb3-4b80-93de-0970fa2786ed admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:49.258329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa0d36-0eb3-4b80-93de-0970fa2786ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:49.258607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa0d36-0eb3-4b80-93de-0970fa2786ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:49.259350 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa0d36-0eb3-4b80-93de-0970fa2786ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:49.264644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa0d36-0eb3-4b80-93de-0970fa2786ed admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:49.322808 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 821/1634] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:35:48 2022] POST /identity/v3/projects => generated 405 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:35:49.326521 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58d81b-e51c-4756-9beb-12cff27f2e21 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:49.346349 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58d81b-e51c-4756-9beb-12cff27f2e21 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:49.468913 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948df6d7-9362-409e-ab92-5ef2befed512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:49.469565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948df6d7-9362-409e-ab92-5ef2befed512 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:49.469847 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948df6d7-9362-409e-ab92-5ef2befed512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:49.470121 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948df6d7-9362-409e-ab92-5ef2befed512 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:49.470803 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948df6d7-9362-409e-ab92-5ef2befed512 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:49.472825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948df6d7-9362-409e-ab92-5ef2befed512 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:49.533734 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 814/1635] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:35:49 2022] POST /identity/v3/users => generated 309 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:35:49.537733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa0d36-0eb3-4b80-93de-0970fa2786ed admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:49.558047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa0d36-0eb3-4b80-93de-0970fa2786ed admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:49.571804 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e61ae6-dcdf-46d9-8884-5e3e2f3afe39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:49.573080 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e61ae6-dcdf-46d9-8884-5e3e2f3afe39 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:49.573567 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e61ae6-dcdf-46d9-8884-5e3e2f3afe39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:49.574003 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e61ae6-dcdf-46d9-8884-5e3e2f3afe39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:49.575215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e61ae6-dcdf-46d9-8884-5e3e2f3afe39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:49.577686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e61ae6-dcdf-46d9-8884-5e3e2f3afe39 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:49.610939 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 822/1636] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:35:49.614645 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948df6d7-9362-409e-ab92-5ef2befed512 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:49.632059 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948df6d7-9362-409e-ab92-5ef2befed512 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:49.646044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d3f65-e978-4b0b-81c4-8ed62f31909a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:49.647428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d3f65-e978-4b0b-81c4-8ed62f31909a admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:49.647929 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d3f65-e978-4b0b-81c4-8ed62f31909a admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:49.648414 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d3f65-e978-4b0b-81c4-8ed62f31909a admin admin] PATH_INFO: `/v3/projects/033aea18564f471faaa2c9b9672eaaeb/users/7e56bf6940fe431db4cb0fd5c58caa7d/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:49.684709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d3f65-e978-4b0b-81c4-8ed62f31909a admin admin] RBAC: Authorizing `identity:create_grant(project_id=033aea18564f471faaa2c9b9672eaaeb, user_id=7e56bf6940fe431db4cb0fd5c58caa7d, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:49.690134 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d3f65-e978-4b0b-81c4-8ed62f31909a admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:49.726559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 815/1637] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:35:49 2022] PUT /identity/v3/projects/033aea18564f471faaa2c9b9672eaaeb/users/7e56bf6940fe431db4cb0fd5c58caa7d/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:35:49.730254 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e61ae6-dcdf-46d9-8884-5e3e2f3afe39 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:49.745053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e61ae6-dcdf-46d9-8884-5e3e2f3afe39 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:49.853414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a02c38-3c93-4121-a828-20e5590bac37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:49.855791 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a02c38-3c93-4121-a828-20e5590bac37 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:49.857071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a02c38-3c93-4121-a828-20e5590bac37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:49.860055 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a02c38-3c93-4121-a828-20e5590bac37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:49.862571 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a02c38-3c93-4121-a828-20e5590bac37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:49.866393 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a02c38-3c93-4121-a828-20e5590bac37 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:49.914036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 823/1638] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:35:49.916933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d3f65-e978-4b0b-81c4-8ed62f31909a admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:49.934442 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d3f65-e978-4b0b-81c4-8ed62f31909a admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:49.952261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f9679-0d9d-4e12-aa6c-553a00a83f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:49.953483 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f9679-0d9d-4e12-aa6c-553a00a83f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:49.954118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f9679-0d9d-4e12-aa6c-553a00a83f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:49.988363 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f9679-0d9d-4e12-aa6c-553a00a83f1a admin admin] PATH_INFO: `/v3/projects/033aea18564f471faaa2c9b9672eaaeb/users/7e56bf6940fe431db4cb0fd5c58caa7d/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:49.988556 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccbddd-3693-4de2-8ac7-da8488422fc1 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:49.988556 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccbddd-3693-4de2-8ac7-da8488422fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:49.988556 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bccbddd-3693-4de2-8ac7-da8488422fc1 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:49.988556 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 824/1639] 10.222.0.74 () {54 vars in 962 bytes} [Wed Jul 27 16:35:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:35:49.988945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f9679-0d9d-4e12-aa6c-553a00a83f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=033aea18564f471faaa2c9b9672eaaeb, user_id=7e56bf6940fe431db4cb0fd5c58caa7d, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:49.988945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f9679-0d9d-4e12-aa6c-553a00a83f1a admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:49.989073 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32052fde-8c8b-467a-94c6-442d13790b58 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:49.989073 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32052fde-8c8b-467a-94c6-442d13790b58 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:49.989073 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32052fde-8c8b-467a-94c6-442d13790b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:50.009514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 816/1640] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:35:49 2022] PUT /identity/v3/projects/033aea18564f471faaa2c9b9672eaaeb/users/7e56bf6940fe431db4cb0fd5c58caa7d/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:35:50.022801 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-32052fde-8c8b-467a-94c6-442d13790b58 None None] MFA Rules not processed for user `56dfafd15e284b7986ba07dd3819bf65`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:50.024092 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-32052fde-8c8b-467a-94c6-442d13790b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:50.111148 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:35:50.203180 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 825/1641] 10.222.0.74 () {58 vars in 1059 bytes} [Wed Jul 27 16:35:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:50.209556 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c967f3e-39bc-40f1-9514-bfa72dd2790c None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:50.210068 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c967f3e-39bc-40f1-9514-bfa72dd2790c None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:50.210496 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c967f3e-39bc-40f1-9514-bfa72dd2790c None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:50.212484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 817/1642] 10.222.0.74 () {54 vars in 912 bytes} [Wed Jul 27 16:35:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:35:50.218425 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b2d32-5b7f-4880-9ce1-0c1763f5c2cb None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:50.218723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b2d32-5b7f-4880-9ce1-0c1763f5c2cb None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:50.219010 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b2d32-5b7f-4880-9ce1-0c1763f5c2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:50.260623 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-902b2d32-5b7f-4880-9ce1-0c1763f5c2cb None None] MFA Rules not processed for user `56dfafd15e284b7986ba07dd3819bf65`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:50.261926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-902b2d32-5b7f-4880-9ce1-0c1763f5c2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:50.273309 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 826/1643] 10.222.0.74 () {58 vars in 1009 bytes} [Wed Jul 27 16:35:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:50.288269 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c967f3e-39bc-40f1-9514-bfa72dd2790c None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:50.353712 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c967f3e-39bc-40f1-9514-bfa72dd2790c None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:50.359383 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384c93e-f982-4c87-bde6-eea33b50b669 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:50.360558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384c93e-f982-4c87-bde6-eea33b50b669 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:50.361097 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384c93e-f982-4c87-bde6-eea33b50b669 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:50.361506 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384c93e-f982-4c87-bde6-eea33b50b669 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:50.385932 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384c93e-f982-4c87-bde6-eea33b50b669 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:50.388075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384c93e-f982-4c87-bde6-eea33b50b669 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:50.415824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 818/1644] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:35:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:35:53.068550 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513560be-75bf-40e9-bb15-7a3b33bc4ab2 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:53.069065 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513560be-75bf-40e9-bb15-7a3b33bc4ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:53.069442 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513560be-75bf-40e9-bb15-7a3b33bc4ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:53.141228 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-513560be-75bf-40e9-bb15-7a3b33bc4ab2 None None] MFA Rules not processed for user `c5710d1f3a6742e489c7c676cc4c01d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:53.142773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-513560be-75bf-40e9-bb15-7a3b33bc4ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:53.309519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 827/1645] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:53.320304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d523641-a3a0-44d1-9fe8-5a121d4f61f7 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:53.320814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d523641-a3a0-44d1-9fe8-5a121d4f61f7 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:53.321072 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d523641-a3a0-44d1-9fe8-5a121d4f61f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:53.331355 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:35:53.367938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-9d523641-a3a0-44d1-9fe8-5a121d4f61f7 None None] MFA Rules not processed for user `c5710d1f3a6742e489c7c676cc4c01d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:53.370241 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9d523641-a3a0-44d1-9fe8-5a121d4f61f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:53.396010 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 819/1646] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:53.408363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513560be-75bf-40e9-bb15-7a3b33bc4ab2 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:53.457009 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513560be-75bf-40e9-bb15-7a3b33bc4ab2 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:53.479721 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb3ec6-970e-4dd9-97d0-fd712faf4a2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:53.481069 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb3ec6-970e-4dd9-97d0-fd712faf4a2a service nova] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:53.481562 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb3ec6-970e-4dd9-97d0-fd712faf4a2a service nova] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:53.482264 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb3ec6-970e-4dd9-97d0-fd712faf4a2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:53.506543 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb3ec6-970e-4dd9-97d0-fd712faf4a2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:53.511731 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb3ec6-970e-4dd9-97d0-fd712faf4a2a service nova] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:53.559710 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 828/1647] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:35:53 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:35:56.227094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98987e18-1b03-4359-b0bc-9a19ddc3de2e None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:56.227655 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98987e18-1b03-4359-b0bc-9a19ddc3de2e None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:56.227655 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98987e18-1b03-4359-b0bc-9a19ddc3de2e None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:56.229292 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 820/1648] 10.222.0.74 () {54 vars in 912 bytes} [Wed Jul 27 16:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:35:56.243977 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717d15a-2ad5-4ebf-9bdc-b1a372ff1f50 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:56.244547 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717d15a-2ad5-4ebf-9bdc-b1a372ff1f50 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:56.244547 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717d15a-2ad5-4ebf-9bdc-b1a372ff1f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:56.356448 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-e717d15a-2ad5-4ebf-9bdc-b1a372ff1f50 None None] MFA Rules not processed for user `193dd9fee1844275ad0dab602e93df11`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:56.361209 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e717d15a-2ad5-4ebf-9bdc-b1a372ff1f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:56.544342 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 829/1649] 10.222.0.74 () {58 vars in 1009 bytes} [Wed Jul 27 16:35:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:56.559249 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025995b-078a-47b5-9a42-5d15c2b7a3c8 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:56.559526 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025995b-078a-47b5-9a42-5d15c2b7a3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:56.560077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025995b-078a-47b5-9a42-5d15c2b7a3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:56.565085 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717d15a-2ad5-4ebf-9bdc-b1a372ff1f50 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:56.581810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717d15a-2ad5-4ebf-9bdc-b1a372ff1f50 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:56.593619 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c93dd-29c7-4a89-94c3-2724485ed247 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:56.594442 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c93dd-29c7-4a89-94c3-2724485ed247 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:56.594962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c93dd-29c7-4a89-94c3-2724485ed247 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:56.595816 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c93dd-29c7-4a89-94c3-2724485ed247 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:56.621269 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c93dd-29c7-4a89-94c3-2724485ed247 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:56.625625 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c93dd-29c7-4a89-94c3-2724485ed247 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:56.631507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-5025995b-078a-47b5-9a42-5d15c2b7a3c8 None None] MFA Rules not processed for user `7e56bf6940fe431db4cb0fd5c58caa7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:56.632934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-5025995b-078a-47b5-9a42-5d15c2b7a3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:56.640154 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 830/1650] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:35:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:35:56.825543 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 821/1651] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:35:56 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:35:58.387132 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7a286-ed38-483a-8f46-97256c391617 None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:58.388246 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7a286-ed38-483a-8f46-97256c391617 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:58.389388 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7a286-ed38-483a-8f46-97256c391617 None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:58.391876 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 831/1652] 10.222.0.74 () {54 vars in 961 bytes} [Wed Jul 27 16:35:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:35:58.400254 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55766d8e-ee9d-4c17-8982-760a74cafba4 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:58.400794 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55766d8e-ee9d-4c17-8982-760a74cafba4 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:58.401260 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55766d8e-ee9d-4c17-8982-760a74cafba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:58.426649 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-55766d8e-ee9d-4c17-8982-760a74cafba4 None None] MFA Rules not processed for user `91f8e0f82a8541aead6cd8cc45f4a86b`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:35:58.428541 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-55766d8e-ee9d-4c17-8982-760a74cafba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:58.638527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 822/1653] 10.222.0.74 () {58 vars in 1058 bytes} [Wed Jul 27 16:35:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:35:58.647532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7a286-ed38-483a-8f46-97256c391617 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:35:58.667315 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7a286-ed38-483a-8f46-97256c391617 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:35:58.692123 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c68db8-3e45-4487-8268-ca991cf1a1df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f8e0f82a8541aead6cd8cc45f4a86b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:35:58.692774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c68db8-3e45-4487-8268-ca991cf1a1df service placement] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:58.693077 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c68db8-3e45-4487-8268-ca991cf1a1df service placement] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:58.693348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c68db8-3e45-4487-8268-ca991cf1a1df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:58.696938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-27c68db8-3e45-4487-8268-ca991cf1a1df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:35:58.723389 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c68db8-3e45-4487-8268-ca991cf1a1df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:35:58.736229 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c68db8-3e45-4487-8268-ca991cf1a1df service placement] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:35:58.782690 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 832/1654] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:35:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:35:59.865367 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2037c-658e-4351-b4af-6fd6837585e3 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:59.865939 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2037c-658e-4351-b4af-6fd6837585e3 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:59.865939 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2037c-658e-4351-b4af-6fd6837585e3 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:59.869136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 823/1655] 10.222.0.74 () {54 vars in 921 bytes} [Wed Jul 27 16:35:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:35:59.958767 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e146527-93f0-4927-8700-d1a82d3afc70 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:35:59.960246 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e146527-93f0-4927-8700-d1a82d3afc70 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:35:59.961919 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e146527-93f0-4927-8700-d1a82d3afc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:35:59.979767 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:00.024119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-6e146527-93f0-4927-8700-d1a82d3afc70 None None] MFA Rules not processed for user `91f8e0f82a8541aead6cd8cc45f4a86b`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:00.026183 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-6e146527-93f0-4927-8700-d1a82d3afc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:00.052994 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 833/1656] 10.222.0.74 () {58 vars in 1018 bytes} [Wed Jul 27 16:35:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:00.094478 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2037c-658e-4351-b4af-6fd6837585e3 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:00.118883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2037c-658e-4351-b4af-6fd6837585e3 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:00.143909 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2143a4-b604-463c-9d26-7ed4ac021ceb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f8e0f82a8541aead6cd8cc45f4a86b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:00.146223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2143a4-b604-463c-9d26-7ed4ac021ceb service placement] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:00.147975 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2143a4-b604-463c-9d26-7ed4ac021ceb service placement] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:00.148703 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2143a4-b604-463c-9d26-7ed4ac021ceb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:00.187199 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2143a4-b604-463c-9d26-7ed4ac021ceb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:00.190563 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2143a4-b604-463c-9d26-7ed4ac021ceb service placement] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:00.221683 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 824/1657] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:36:00.476102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e146527-93f0-4927-8700-d1a82d3afc70 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:00.692807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e146527-93f0-4927-8700-d1a82d3afc70 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:00.707332 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca697101-0e5b-41e2-8d0d-b2cac03cd4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:00.708067 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca697101-0e5b-41e2-8d0d-b2cac03cd4be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:00.708355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca697101-0e5b-41e2-8d0d-b2cac03cd4be admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:00.708677 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca697101-0e5b-41e2-8d0d-b2cac03cd4be admin admin] PATH_INFO: `/v3/users/2cc374498a024bd184d024308cc313e8` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:00.712626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca697101-0e5b-41e2-8d0d-b2cac03cd4be admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cc374498a024bd184d024308cc313e8)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:00.714860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca697101-0e5b-41e2-8d0d-b2cac03cd4be admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:00.859276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ca697101-0e5b-41e2-8d0d-b2cac03cd4be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cc374498a024bd184d024308cc313e8'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:00.876397 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ca697101-0e5b-41e2-8d0d-b2cac03cd4be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cc374498a024bd184d024308cc313e8'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:00.940003 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ca697101-0e5b-41e2-8d0d-b2cac03cd4be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cc374498a024bd184d024308cc313e8'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:00.962281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ca697101-0e5b-41e2-8d0d-b2cac03cd4be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cc374498a024bd184d024308cc313e8'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:00.965233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ca697101-0e5b-41e2-8d0d-b2cac03cd4be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cc374498a024bd184d024308cc313e8'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:00.996204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 834/1658] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:00 2022] DELETE /identity/v3/users/2cc374498a024bd184d024308cc313e8 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:01.005220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ce59d-c49b-4998-b795-a44b4e41f322 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:01.005735 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ce59d-c49b-4998-b795-a44b4e41f322 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:01.006161 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ce59d-c49b-4998-b795-a44b4e41f322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:01.063021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-488ce59d-c49b-4998-b795-a44b4e41f322 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:01.064432 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-488ce59d-c49b-4998-b795-a44b4e41f322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:01.299741 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 825/1659] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:01.373368 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca697101-0e5b-41e2-8d0d-b2cac03cd4be admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:01.375556 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-ca697101-0e5b-41e2-8d0d-b2cac03cd4be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:01.571273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca697101-0e5b-41e2-8d0d-b2cac03cd4be admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:01.575868 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990d35c-131e-4e96-9139-3c7acf4afafb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:01.576834 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990d35c-131e-4e96-9139-3c7acf4afafb service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:01.577284 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990d35c-131e-4e96-9139-3c7acf4afafb service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:01.577716 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990d35c-131e-4e96-9139-3c7acf4afafb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:01.592933 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f990d35c-131e-4e96-9139-3c7acf4afafb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:01.595014 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f990d35c-131e-4e96-9139-3c7acf4afafb service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:01.612529 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 835/1660] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:01.891503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19cf95-ac0a-4712-bcad-4ab5f3e64862 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:01.891659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19cf95-ac0a-4712-bcad-4ab5f3e64862 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:01.892208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19cf95-ac0a-4712-bcad-4ab5f3e64862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:01.917260 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:01.963920 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-4c19cf95-ac0a-4712-bcad-4ab5f3e64862 None None] MFA Rules not processed for user `22bb7775bae64374b11462a53853e54f`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:01.966435 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-4c19cf95-ac0a-4712-bcad-4ab5f3e64862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:01.984387 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212273a-247a-4276-b0f1-9077981e5863 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:01.985402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212273a-247a-4276-b0f1-9077981e5863 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:01.985992 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212273a-247a-4276-b0f1-9077981e5863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:02.082597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:02.150348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-6212273a-247a-4276-b0f1-9077981e5863 None None] MFA Rules not processed for user `07d6bffc69674103b4c6540598a780b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:02.151633 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-6212273a-247a-4276-b0f1-9077981e5863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:02.188067 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 826/1661] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:36:02.205123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19cf95-ac0a-4712-bcad-4ab5f3e64862 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:02.206669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-4c19cf95-ac0a-4712-bcad-4ab5f3e64862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:02.355669 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 836/1662] 10.222.0.74 () {58 vars in 1009 bytes} [Wed Jul 27 16:36:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:02.418787 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19cf95-ac0a-4712-bcad-4ab5f3e64862 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:02.435834 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff39ab-c526-4634-94f9-0e08bc2ab12f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:02.438923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff39ab-c526-4634-94f9-0e08bc2ab12f service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:02.440475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff39ab-c526-4634-94f9-0e08bc2ab12f service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:02.440773 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff39ab-c526-4634-94f9-0e08bc2ab12f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:02.471215 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff39ab-c526-4634-94f9-0e08bc2ab12f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:02.478605 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff39ab-c526-4634-94f9-0e08bc2ab12f service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:02.535621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 827/1663] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:36:02.667149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212273a-247a-4276-b0f1-9077981e5863 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:02.671710 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-6212273a-247a-4276-b0f1-9077981e5863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:02.970268 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212273a-247a-4276-b0f1-9077981e5863 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:02.976780 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1572795-ff33-4ec3-a3ce-e480f8197d5f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb45ded9f75e4fa2973b564f83bdfaf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:02.977546 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1572795-ff33-4ec3-a3ce-e480f8197d5f service swift] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:02.977843 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1572795-ff33-4ec3-a3ce-e480f8197d5f service swift] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:02.978121 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1572795-ff33-4ec3-a3ce-e480f8197d5f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:02.993820 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1572795-ff33-4ec3-a3ce-e480f8197d5f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:02.996930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1572795-ff33-4ec3-a3ce-e480f8197d5f service swift] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:03.030800 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 837/1664] 10.222.0.74 () {60 vars in 1384 bytes} [Wed Jul 27 16:36:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 16:36:03.264039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336f91f-e18a-4b38-ba2a-a2099fcc4711 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:03.264516 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336f91f-e18a-4b38-ba2a-a2099fcc4711 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:03.264622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336f91f-e18a-4b38-ba2a-a2099fcc4711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:03.374167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-c336f91f-e18a-4b38-ba2a-a2099fcc4711 None None] MFA Rules not processed for user `140059c5b30c426ab6d7f1236eb5f684`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:03.375421 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c336f91f-e18a-4b38-ba2a-a2099fcc4711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:03.704038 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 828/1665] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:03.715713 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1572795-ff33-4ec3-a3ce-e480f8197d5f service swift] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:03.743632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1572795-ff33-4ec3-a3ce-e480f8197d5f service swift] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:03.771993 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b6ae9-4d18-4937-9cc1-c150ca80cb00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:03.772765 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b6ae9-4d18-4937-9cc1-c150ca80cb00 service nova] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:03.776324 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b6ae9-4d18-4937-9cc1-c150ca80cb00 service nova] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:03.779216 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b6ae9-4d18-4937-9cc1-c150ca80cb00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:03.821963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b6ae9-4d18-4937-9cc1-c150ca80cb00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:03.830915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b6ae9-4d18-4937-9cc1-c150ca80cb00 service nova] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:03.872942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 838/1666] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:04.306542 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336f91f-e18a-4b38-ba2a-a2099fcc4711 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:04.308070 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c336f91f-e18a-4b38-ba2a-a2099fcc4711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:04.354039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336f91f-e18a-4b38-ba2a-a2099fcc4711 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:04.381130 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3a7da-fa2a-4da4-931c-23c753e1531f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:04.381810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3a7da-fa2a-4da4-931c-23c753e1531f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:04.382095 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3a7da-fa2a-4da4-931c-23c753e1531f admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:04.382373 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3a7da-fa2a-4da4-931c-23c753e1531f admin admin] PATH_INFO: `/v3/projects/7a3e43f9afcd49ce89b5911eef170bce` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:04.392727 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55383510-e8f7-49f8-8073-12e8e933f4f7 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:04.393340 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55383510-e8f7-49f8-8073-12e8e933f4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:04.393340 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55383510-e8f7-49f8-8073-12e8e933f4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:04.398773 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3a7da-fa2a-4da4-931c-23c753e1531f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a3e43f9afcd49ce89b5911eef170bce)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:04.417338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3a7da-fa2a-4da4-931c-23c753e1531f admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:04.433679 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:04.484060 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-55383510-e8f7-49f8-8073-12e8e933f4f7 None None] MFA Rules not processed for user `7a0bd78165c94d9e9750fe4af4f7d375`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:04.486126 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-55383510-e8f7-49f8-8073-12e8e933f4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:04.675034 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b8e3a7da-fa2a-4da4-931c-23c753e1531f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a3e43f9afcd49ce89b5911eef170bce', 'request_id': 'req-b8e3a7da-fa2a-4da4-931c-23c753e1531f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:04.679295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:04.707834 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b8e3a7da-fa2a-4da4-931c-23c753e1531f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a3e43f9afcd49ce89b5911eef170bce', 'request_id': 'req-b8e3a7da-fa2a-4da4-931c-23c753e1531f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:04.744858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b8e3a7da-fa2a-4da4-931c-23c753e1531f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a3e43f9afcd49ce89b5911eef170bce', 'request_id': 'req-b8e3a7da-fa2a-4da4-931c-23c753e1531f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:04.813869 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 839/1667] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:04.820182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55383510-e8f7-49f8-8073-12e8e933f4f7 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:04.831841 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b8e3a7da-fa2a-4da4-931c-23c753e1531f admin admin] The token cache is being invalidate because project 7a3e43f9afcd49ce89b5911eef170bce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:04.847884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b8e3a7da-fa2a-4da4-931c-23c753e1531f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:04.855219 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b8e3a7da-fa2a-4da4-931c-23c753e1531f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:04.861514 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55383510-e8f7-49f8-8073-12e8e933f4f7 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:04.863539 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 829/1668] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:04 2022] DELETE /identity/v3/projects/7a3e43f9afcd49ce89b5911eef170bce => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:04.868171 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf31a7-0008-4be2-95d1-6120e5a7ba77 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:04.868476 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf31a7-0008-4be2-95d1-6120e5a7ba77 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:04.868760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf31a7-0008-4be2-95d1-6120e5a7ba77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:05.002306 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-59cf31a7-0008-4be2-95d1-6120e5a7ba77 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:05.006672 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-59cf31a7-0008-4be2-95d1-6120e5a7ba77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:05.080098 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5b8aa-ae96-4d91-b77a-6c1118097a7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:05.080970 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5b8aa-ae96-4d91-b77a-6c1118097a7a service nova] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:05.081370 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5b8aa-ae96-4d91-b77a-6c1118097a7a service nova] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:05.081759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5b8aa-ae96-4d91-b77a-6c1118097a7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:05.086940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-53f5b8aa-ae96-4d91-b77a-6c1118097a7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:05.339207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f5b8aa-ae96-4d91-b77a-6c1118097a7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:05.343376 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f5b8aa-ae96-4d91-b77a-6c1118097a7a service nova] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:05.373743 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 830/1669] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 506 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:05.381022 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bfa41-c766-4a1e-829b-5b51c1a104b4 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:05.381589 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bfa41-c766-4a1e-829b-5b51c1a104b4 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:05.382050 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bfa41-c766-4a1e-829b-5b51c1a104b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:05.393540 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 840/1670] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:05.458490 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-235bfa41-c766-4a1e-829b-5b51c1a104b4 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:05.464552 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-235bfa41-c766-4a1e-829b-5b51c1a104b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:05.495814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 831/1671] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:05.500490 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5b8aa-ae96-4d91-b77a-6c1118097a7a service nova] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:05.512556 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5b8aa-ae96-4d91-b77a-6c1118097a7a service nova] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:05.530938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd438127-c7e5-4291-aa5c-18da8fd6d96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:05.531525 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd438127-c7e5-4291-aa5c-18da8fd6d96c admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:05.532108 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd438127-c7e5-4291-aa5c-18da8fd6d96c admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:05.532395 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd438127-c7e5-4291-aa5c-18da8fd6d96c admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:05.533142 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd438127-c7e5-4291-aa5c-18da8fd6d96c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:36:05.533454 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd438127-c7e5-4291-aa5c-18da8fd6d96c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:05.535128 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd438127-c7e5-4291-aa5c-18da8fd6d96c admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:05.581821 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 841/1672] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:36:05.587185 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235bfa41-c766-4a1e-829b-5b51c1a104b4 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:05.608018 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235bfa41-c766-4a1e-829b-5b51c1a104b4 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:05.628766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d50f9-2192-4aed-877b-b3d839acdb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:05.631073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d50f9-2192-4aed-877b-b3d839acdb4d admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:05.633906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d50f9-2192-4aed-877b-b3d839acdb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:05.636007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d50f9-2192-4aed-877b-b3d839acdb4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:05.636824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d50f9-2192-4aed-877b-b3d839acdb4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:05.644128 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d50f9-2192-4aed-877b-b3d839acdb4d admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:05.797936 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 832/1673] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:05 2022] POST /identity/v3/projects => generated 357 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:05.799501 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd438127-c7e5-4291-aa5c-18da8fd6d96c admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:05.816961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd438127-c7e5-4291-aa5c-18da8fd6d96c admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:06.115235 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c3c51-21d0-405b-8d01-1e35706ee059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:06.116603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c3c51-21d0-405b-8d01-1e35706ee059 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:06.117122 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c3c51-21d0-405b-8d01-1e35706ee059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:06.117748 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c3c51-21d0-405b-8d01-1e35706ee059 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:06.130603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c3c51-21d0-405b-8d01-1e35706ee059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:06.135807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c3c51-21d0-405b-8d01-1e35706ee059 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:06.205519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:06.262483 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d50f9-2192-4aed-877b-b3d839acdb4d admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:06.263830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 842/1674] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:05 2022] POST /identity/v3/users => generated 279 bytes in 465 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:06.286999 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d50f9-2192-4aed-877b-b3d839acdb4d admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:06.306050 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286fc8b-41c6-4ec5-b1e4-ee28ea4514ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:06.306726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286fc8b-41c6-4ec5-b1e4-ee28ea4514ae admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:06.307146 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286fc8b-41c6-4ec5-b1e4-ee28ea4514ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:06.307257 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286fc8b-41c6-4ec5-b1e4-ee28ea4514ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:06.309089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286fc8b-41c6-4ec5-b1e4-ee28ea4514ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:06.318353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286fc8b-41c6-4ec5-b1e4-ee28ea4514ae admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:06.384544 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 833/1675] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:36:06.388661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c3c51-21d0-405b-8d01-1e35706ee059 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:06.410043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c3c51-21d0-405b-8d01-1e35706ee059 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:06.425573 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a946900-46d7-40e6-a4d0-da2cb8390fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:06.426519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a946900-46d7-40e6-a4d0-da2cb8390fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:06.427021 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a946900-46d7-40e6-a4d0-da2cb8390fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:06.427501 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a946900-46d7-40e6-a4d0-da2cb8390fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:06.429634 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a946900-46d7-40e6-a4d0-da2cb8390fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:06.434252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a946900-46d7-40e6-a4d0-da2cb8390fd2 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:06.495285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 843/1676] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:06.500098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286fc8b-41c6-4ec5-b1e4-ee28ea4514ae admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:06.518287 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286fc8b-41c6-4ec5-b1e4-ee28ea4514ae admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:06.538567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba91f6-77c6-4812-a7ee-d87f5076cb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:06.540276 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba91f6-77c6-4812-a7ee-d87f5076cb6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:06.541283 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba91f6-77c6-4812-a7ee-d87f5076cb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:06.544761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba91f6-77c6-4812-a7ee-d87f5076cb6c admin admin] PATH_INFO: `/v3/projects/62153ff309e1417eb182db62eaa5b48b/users/4552b96c7500492ea4dc047c6dcae3ab/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:06.617058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba91f6-77c6-4812-a7ee-d87f5076cb6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=62153ff309e1417eb182db62eaa5b48b, user_id=4552b96c7500492ea4dc047c6dcae3ab, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:06.638654 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba91f6-77c6-4812-a7ee-d87f5076cb6c admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:06.695628 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 834/1677] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:06 2022] PUT /identity/v3/projects/62153ff309e1417eb182db62eaa5b48b/users/4552b96c7500492ea4dc047c6dcae3ab/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:36:06.708081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c1246-1a75-4933-b194-da0bff6b92f2 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:06.708583 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c1246-1a75-4933-b194-da0bff6b92f2 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:06.708985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c1246-1a75-4933-b194-da0bff6b92f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:06.800675 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-947c1246-1a75-4933-b194-da0bff6b92f2 None None] MFA Rules not processed for user `4552b96c7500492ea4dc047c6dcae3ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:06.802102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-947c1246-1a75-4933-b194-da0bff6b92f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:06.969756 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d12a7-f136-4db4-b0b8-e3546483f8ee None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:06.970648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d12a7-f136-4db4-b0b8-e3546483f8ee None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:06.970937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d12a7-f136-4db4-b0b8-e3546483f8ee None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:06.973258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 835/1678] 10.222.0.74 () {54 vars in 921 bytes} [Wed Jul 27 16:36:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:36:07.002205 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d12a7-f136-4db4-b0b8-e3546483f8ee None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:07.004668 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-3e7d12a7-f136-4db4-b0b8-e3546483f8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:07.024461 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 844/1679] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:06 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:07.029458 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c1246-1a75-4933-b194-da0bff6b92f2 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:07.105991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:07.203956 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d12a7-f136-4db4-b0b8-e3546483f8ee None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:07.218058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba92fb-cfd4-46bd-a82b-46ef8f24b66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:07.219623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba92fb-cfd4-46bd-a82b-46ef8f24b66d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:07.221456 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba92fb-cfd4-46bd-a82b-46ef8f24b66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:07.222473 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba92fb-cfd4-46bd-a82b-46ef8f24b66d admin admin] PATH_INFO: `/v3/users/92130afa772e4d6e9b3c018eb2687712` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:07.228064 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba92fb-cfd4-46bd-a82b-46ef8f24b66d admin admin] RBAC: Authorizing `identity:delete_user(user_id=92130afa772e4d6e9b3c018eb2687712)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:07.229536 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: WARNING py.warnings [None req-2bba92fb-cfd4-46bd-a82b-46ef8f24b66d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 16:36:07.229536 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: warnings.warn(msg) Jul 27 16:36:07.229536 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: Jul 27 16:36:07.233740 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba92fb-cfd4-46bd-a82b-46ef8f24b66d admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:07.298922 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c1246-1a75-4933-b194-da0bff6b92f2 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:07.312622 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4792b4-ed61-4b60-9d60-5dd7e8eb615b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:07.313870 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4792b4-ed61-4b60-9d60-5dd7e8eb615b admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:07.314214 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4792b4-ed61-4b60-9d60-5dd7e8eb615b admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:07.314504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4792b4-ed61-4b60-9d60-5dd7e8eb615b admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:07.315187 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4792b4-ed61-4b60-9d60-5dd7e8eb615b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:07.320088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4792b4-ed61-4b60-9d60-5dd7e8eb615b admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:07.417978 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-2bba92fb-cfd4-46bd-a82b-46ef8f24b66d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92130afa772e4d6e9b3c018eb2687712'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:07.429995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 845/1680] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:07 2022] POST /identity/v3/projects => generated 355 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:36:07.432930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f93e77-91b2-4aa8-b156-1d6083152655 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:07.435026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-2bba92fb-cfd4-46bd-a82b-46ef8f24b66d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92130afa772e4d6e9b3c018eb2687712'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:07.436269 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f93e77-91b2-4aa8-b156-1d6083152655 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:07.439055 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f93e77-91b2-4aa8-b156-1d6083152655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:07.465972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-2bba92fb-cfd4-46bd-a82b-46ef8f24b66d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92130afa772e4d6e9b3c018eb2687712'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:07.518769 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-03f93e77-91b2-4aa8-b156-1d6083152655 None None] MFA Rules not processed for user `91f8e0f82a8541aead6cd8cc45f4a86b`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:07.520047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-03f93e77-91b2-4aa8-b156-1d6083152655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:07.522645 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-2bba92fb-cfd4-46bd-a82b-46ef8f24b66d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92130afa772e4d6e9b3c018eb2687712'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:07.524976 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-2bba92fb-cfd4-46bd-a82b-46ef8f24b66d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92130afa772e4d6e9b3c018eb2687712'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:07.546292 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 836/1681] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:07 2022] DELETE /identity/v3/users/92130afa772e4d6e9b3c018eb2687712 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:36:07.548261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba92fb-cfd4-46bd-a82b-46ef8f24b66d admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:07.553503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-2bba92fb-cfd4-46bd-a82b-46ef8f24b66d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:07.705417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:07.724982 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba92fb-cfd4-46bd-a82b-46ef8f24b66d admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:07.738959 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e37e4-5f35-443a-9ba9-aed5555e5f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:07.741461 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e37e4-5f35-443a-9ba9-aed5555e5f08 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:07.741461 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e37e4-5f35-443a-9ba9-aed5555e5f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:07.741461 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e37e4-5f35-443a-9ba9-aed5555e5f08 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:07.743419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e37e4-5f35-443a-9ba9-aed5555e5f08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:07.749471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e37e4-5f35-443a-9ba9-aed5555e5f08 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:07.762232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 846/1682] 10.222.0.74 () {58 vars in 1018 bytes} [Wed Jul 27 16:36:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:07.764001 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa1eee-070f-4b2e-9146-5e90f17f0c49 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:07.764432 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa1eee-070f-4b2e-9146-5e90f17f0c49 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:07.764723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa1eee-070f-4b2e-9146-5e90f17f0c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:07.809266 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 837/1683] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:07 2022] POST /identity/v3/users => generated 284 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:07.810541 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e37e4-5f35-443a-9ba9-aed5555e5f08 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:07.817457 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b89e37e4-5f35-443a-9ba9-aed5555e5f08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:07.832310 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-95fa1eee-070f-4b2e-9146-5e90f17f0c49 None None] MFA Rules not processed for user `193dd9fee1844275ad0dab602e93df11`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:07.834351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-95fa1eee-070f-4b2e-9146-5e90f17f0c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:07.959938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:08.003190 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 847/1684] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Jul 27 16:36:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:08.005207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa1eee-070f-4b2e-9146-5e90f17f0c49 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:08.029107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e37e4-5f35-443a-9ba9-aed5555e5f08 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:08.041192 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7e504-0d9f-4829-92be-9b9314e6909e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f8e0f82a8541aead6cd8cc45f4a86b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:08.041617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa1eee-070f-4b2e-9146-5e90f17f0c49 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:08.042317 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7e504-0d9f-4829-92be-9b9314e6909e service placement] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:08.043848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7e504-0d9f-4829-92be-9b9314e6909e service placement] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:08.045125 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7e504-0d9f-4829-92be-9b9314e6909e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:08.059931 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc41b31-88e4-4793-89a2-2257e5707788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:08.060680 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc41b31-88e4-4793-89a2-2257e5707788 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:08.061019 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc41b31-88e4-4793-89a2-2257e5707788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:08.062714 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc41b31-88e4-4793-89a2-2257e5707788 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:08.065869 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc41b31-88e4-4793-89a2-2257e5707788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:08.075641 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc41b31-88e4-4793-89a2-2257e5707788 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:08.075755 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a7e504-0d9f-4829-92be-9b9314e6909e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:08.095979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a7e504-0d9f-4829-92be-9b9314e6909e service placement] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:08.115625 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 848/1685] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:08.117910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc41b31-88e4-4793-89a2-2257e5707788 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:08.131431 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 838/1686] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:08.161039 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc41b31-88e4-4793-89a2-2257e5707788 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:08.188179 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecf5a9-f1fb-43b2-a8a5-5daa3f03f288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:08.188918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecf5a9-f1fb-43b2-a8a5-5daa3f03f288 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:08.189196 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecf5a9-f1fb-43b2-a8a5-5daa3f03f288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:08.191333 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecf5a9-f1fb-43b2-a8a5-5daa3f03f288 admin admin] PATH_INFO: `/v3/projects/9cb1b317b9bf4c7fa19c3720ac4986d5/users/c4ae96bd492243b4abc87b70c3a858cc/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:08.239412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecf5a9-f1fb-43b2-a8a5-5daa3f03f288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cb1b317b9bf4c7fa19c3720ac4986d5, user_id=c4ae96bd492243b4abc87b70c3a858cc, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:08.252174 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecf5a9-f1fb-43b2-a8a5-5daa3f03f288 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:08.274461 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7e504-0d9f-4829-92be-9b9314e6909e service placement] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:08.276716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-f2a7e504-0d9f-4829-92be-9b9314e6909e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:08.290052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 849/1687] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:08 2022] PUT /identity/v3/projects/9cb1b317b9bf4c7fa19c3720ac4986d5/users/c4ae96bd492243b4abc87b70c3a858cc/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:08.295814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecf5a9-f1fb-43b2-a8a5-5daa3f03f288 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:08.304133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7e504-0d9f-4829-92be-9b9314e6909e service placement] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:08.463885 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21bdd0-92e4-40de-b6ad-3909a5c05564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:08.464783 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21bdd0-92e4-40de-b6ad-3909a5c05564 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:08.465696 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21bdd0-92e4-40de-b6ad-3909a5c05564 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:08.465696 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21bdd0-92e4-40de-b6ad-3909a5c05564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:08.485090 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21bdd0-92e4-40de-b6ad-3909a5c05564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:08.488667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21bdd0-92e4-40de-b6ad-3909a5c05564 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:08.509934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecf5a9-f1fb-43b2-a8a5-5daa3f03f288 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:08.519251 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5323b6-2519-44b7-a69c-24542759bca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:08.520335 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5323b6-2519-44b7-a69c-24542759bca6 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:08.520881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5323b6-2519-44b7-a69c-24542759bca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:08.521395 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5323b6-2519-44b7-a69c-24542759bca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:08.523008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 839/1688] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:08.523605 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5323b6-2519-44b7-a69c-24542759bca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:08.526527 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5323b6-2519-44b7-a69c-24542759bca6 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:08.549329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 850/1689] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:08.550691 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21bdd0-92e4-40de-b6ad-3909a5c05564 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:08.575016 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21bdd0-92e4-40de-b6ad-3909a5c05564 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:08.589369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb108787-aba6-412b-bf77-2f00816f69af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:08.590074 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb108787-aba6-412b-bf77-2f00816f69af admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:08.590356 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb108787-aba6-412b-bf77-2f00816f69af admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:08.590816 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb108787-aba6-412b-bf77-2f00816f69af admin admin] PATH_INFO: `/v3/projects/9cb1b317b9bf4c7fa19c3720ac4986d5/users/c4ae96bd492243b4abc87b70c3a858cc/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:08.603205 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb108787-aba6-412b-bf77-2f00816f69af admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cb1b317b9bf4c7fa19c3720ac4986d5, user_id=c4ae96bd492243b4abc87b70c3a858cc, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:08.613043 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb108787-aba6-412b-bf77-2f00816f69af admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:08.636359 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 840/1690] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:08 2022] PUT /identity/v3/projects/9cb1b317b9bf4c7fa19c3720ac4986d5/users/c4ae96bd492243b4abc87b70c3a858cc/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:08.650524 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602e3aa-f27b-40b8-85fd-b47a3c8bea4a None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:08.650990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602e3aa-f27b-40b8-85fd-b47a3c8bea4a None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:08.651100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602e3aa-f27b-40b8-85fd-b47a3c8bea4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:08.723638 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-d602e3aa-f27b-40b8-85fd-b47a3c8bea4a None None] MFA Rules not processed for user `c4ae96bd492243b4abc87b70c3a858cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:08.724979 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d602e3aa-f27b-40b8-85fd-b47a3c8bea4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:08.833492 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1ea18-8ef3-40d3-956f-82dace81892e None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:08.833582 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1ea18-8ef3-40d3-956f-82dace81892e None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:08.833876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1ea18-8ef3-40d3-956f-82dace81892e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:08.864153 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:08.918638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-0fa1ea18-8ef3-40d3-956f-82dace81892e None None] MFA Rules not processed for user `1c0838069cba4d1389ec6fc809db70a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:08.919906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0fa1ea18-8ef3-40d3-956f-82dace81892e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:08.928695 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 851/1691] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:08 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:08.939517 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602e3aa-f27b-40b8-85fd-b47a3c8bea4a None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:08.941265 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d602e3aa-f27b-40b8-85fd-b47a3c8bea4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:09.158678 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602e3aa-f27b-40b8-85fd-b47a3c8bea4a None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:09.172343 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d763fd3-5d9c-4265-87a0-2cfba4caaa98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:09.173075 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d763fd3-5d9c-4265-87a0-2cfba4caaa98 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:09.173361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d763fd3-5d9c-4265-87a0-2cfba4caaa98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:09.173635 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d763fd3-5d9c-4265-87a0-2cfba4caaa98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:09.174232 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 841/1692] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:09.182102 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d2bd8-a461-4061-9777-14140255b92f None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:09.182403 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d2bd8-a461-4061-9777-14140255b92f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:09.182684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d2bd8-a461-4061-9777-14140255b92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:09.210043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d763fd3-5d9c-4265-87a0-2cfba4caaa98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:09.211854 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d763fd3-5d9c-4265-87a0-2cfba4caaa98 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:09.247377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-cc7d2bd8-a461-4061-9777-14140255b92f None None] MFA Rules not processed for user `1c0838069cba4d1389ec6fc809db70a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:09.248663 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-cc7d2bd8-a461-4061-9777-14140255b92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:09.260881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 852/1693] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:08 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:09.279140 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 842/1694] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:36:09.291707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d763fd3-5d9c-4265-87a0-2cfba4caaa98 service cinder] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:09.293481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-5d763fd3-5d9c-4265-87a0-2cfba4caaa98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:09.514926 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d2bd8-a461-4061-9777-14140255b92f None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:09.517353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-cc7d2bd8-a461-4061-9777-14140255b92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:09.551131 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d2bd8-a461-4061-9777-14140255b92f None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:09.577779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c491798-6e04-4ae7-9795-0aa006e174d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:09.578613 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c491798-6e04-4ae7-9795-0aa006e174d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:09.580824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c491798-6e04-4ae7-9795-0aa006e174d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:09.581298 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c491798-6e04-4ae7-9795-0aa006e174d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:09.589007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-3c491798-6e04-4ae7-9795-0aa006e174d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:09.612036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d763fd3-5d9c-4265-87a0-2cfba4caaa98 service cinder] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:09.643232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb999f-ffba-4121-a8f8-876abc89303a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:09.644105 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb999f-ffba-4121-a8f8-876abc89303a service nova] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:09.646293 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb999f-ffba-4121-a8f8-876abc89303a service nova] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:09.646823 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb999f-ffba-4121-a8f8-876abc89303a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:09.672799 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb999f-ffba-4121-a8f8-876abc89303a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:09.674832 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb999f-ffba-4121-a8f8-876abc89303a service nova] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:09.713221 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 853/1695] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:09.726319 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:09.782077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c491798-6e04-4ae7-9795-0aa006e174d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:09.784014 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c491798-6e04-4ae7-9795-0aa006e174d8 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:09.817363 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 843/1696] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:09 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:10.387046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb999f-ffba-4121-a8f8-876abc89303a service nova] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:10.389692 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-33cb999f-ffba-4121-a8f8-876abc89303a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:10.653849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb999f-ffba-4121-a8f8-876abc89303a service nova] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:10.680466 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f762ce-781d-4720-840a-bf19aeb22bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:10.683757 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f762ce-781d-4720-840a-bf19aeb22bdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:10.684322 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f762ce-781d-4720-840a-bf19aeb22bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:10.685457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f762ce-781d-4720-840a-bf19aeb22bdc admin admin] PATH_INFO: `/v3/projects/2e682c25fdf54f80bbb9fbce41d92b77` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:10.691027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f762ce-781d-4720-840a-bf19aeb22bdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e682c25fdf54f80bbb9fbce41d92b77)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:10.693129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: WARNING py.warnings [None req-90f762ce-781d-4720-840a-bf19aeb22bdc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 16:36:10.693129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: warnings.warn(msg) Jul 27 16:36:10.693129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: Jul 27 16:36:10.700290 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f762ce-781d-4720-840a-bf19aeb22bdc admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:10.802300 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:10.900759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-90f762ce-781d-4720-840a-bf19aeb22bdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e682c25fdf54f80bbb9fbce41d92b77', 'request_id': 'req-90f762ce-781d-4720-840a-bf19aeb22bdc'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:10.930284 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-90f762ce-781d-4720-840a-bf19aeb22bdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e682c25fdf54f80bbb9fbce41d92b77', 'request_id': 'req-90f762ce-781d-4720-840a-bf19aeb22bdc'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:10.950010 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-90f762ce-781d-4720-840a-bf19aeb22bdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e682c25fdf54f80bbb9fbce41d92b77', 'request_id': 'req-90f762ce-781d-4720-840a-bf19aeb22bdc'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:11.016522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-90f762ce-781d-4720-840a-bf19aeb22bdc admin admin] The token cache is being invalidate because project 2e682c25fdf54f80bbb9fbce41d92b77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:11.016834 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-90f762ce-781d-4720-840a-bf19aeb22bdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:11.019968 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-90f762ce-781d-4720-840a-bf19aeb22bdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:11.023617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 854/1697] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:10 2022] DELETE /identity/v3/projects/2e682c25fdf54f80bbb9fbce41d92b77 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:11.028195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c491798-6e04-4ae7-9795-0aa006e174d8 service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:11.055295 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c491798-6e04-4ae7-9795-0aa006e174d8 service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:11.200285 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c0347-5304-45ec-954f-cea3771e6b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:11.202379 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c0347-5304-45ec-954f-cea3771e6b64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:11.206312 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c0347-5304-45ec-954f-cea3771e6b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:11.206823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c0347-5304-45ec-954f-cea3771e6b64 admin admin] PATH_INFO: `/v3/users/7e56bf6940fe431db4cb0fd5c58caa7d` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:11.209904 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c0347-5304-45ec-954f-cea3771e6b64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e56bf6940fe431db4cb0fd5c58caa7d)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:11.219664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c0347-5304-45ec-954f-cea3771e6b64 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:11.443373 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-917c0347-5304-45ec-954f-cea3771e6b64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e56bf6940fe431db4cb0fd5c58caa7d'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:11.470046 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-917c0347-5304-45ec-954f-cea3771e6b64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e56bf6940fe431db4cb0fd5c58caa7d'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:11.514613 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-917c0347-5304-45ec-954f-cea3771e6b64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e56bf6940fe431db4cb0fd5c58caa7d'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:11.576932 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-917c0347-5304-45ec-954f-cea3771e6b64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e56bf6940fe431db4cb0fd5c58caa7d'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:11.579718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-917c0347-5304-45ec-954f-cea3771e6b64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e56bf6940fe431db4cb0fd5c58caa7d'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:11.602728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 844/1698] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:11 2022] DELETE /identity/v3/users/7e56bf6940fe431db4cb0fd5c58caa7d => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:11.943305 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77286e-38e6-4b85-b65b-33e9b7e52f3f None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:11.944747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77286e-38e6-4b85-b65b-33e9b7e52f3f None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:11.945214 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77286e-38e6-4b85-b65b-33e9b7e52f3f None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:11.947384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 855/1699] 10.222.0.74 () {54 vars in 921 bytes} [Wed Jul 27 16:36:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:36:12.121312 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606f13f-5241-4e8e-b874-daa1855637bd None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:12.121930 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606f13f-5241-4e8e-b874-daa1855637bd None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:12.121930 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606f13f-5241-4e8e-b874-daa1855637bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:12.215748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-6606f13f-5241-4e8e-b874-daa1855637bd None None] MFA Rules not processed for user `56dfafd15e284b7986ba07dd3819bf65`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:12.221933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6606f13f-5241-4e8e-b874-daa1855637bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:12.281697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddad7b-06b4-474a-b0ec-4dcf5b529b0a None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:12.281939 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddad7b-06b4-474a-b0ec-4dcf5b529b0a None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:12.282228 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddad7b-06b4-474a-b0ec-4dcf5b529b0a None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:12.284109 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 856/1700] 10.222.0.74 () {54 vars in 912 bytes} [Wed Jul 27 16:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:36:12.289575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bf12a-4ee3-4ea6-a4ca-370d45fff47b None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:12.289883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bf12a-4ee3-4ea6-a4ca-370d45fff47b None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:12.290392 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bf12a-4ee3-4ea6-a4ca-370d45fff47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:12.422714 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-f39bf12a-4ee3-4ea6-a4ca-370d45fff47b None None] MFA Rules not processed for user `193dd9fee1844275ad0dab602e93df11`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:12.423992 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f39bf12a-4ee3-4ea6-a4ca-370d45fff47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:12.565844 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:12.578164 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 857/1701] 10.222.0.74 () {58 vars in 1009 bytes} [Wed Jul 27 16:36:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:12.645289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 845/1702] 10.222.0.74 () {58 vars in 1018 bytes} [Wed Jul 27 16:36:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 525 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:12.829237 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39bf12a-4ee3-4ea6-a4ca-370d45fff47b None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:12.830765 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f39bf12a-4ee3-4ea6-a4ca-370d45fff47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:12.849860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39bf12a-4ee3-4ea6-a4ca-370d45fff47b None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:12.890685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7e176-0f31-4703-8336-51ccb1e4c529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:12.891448 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7e176-0f31-4703-8336-51ccb1e4c529 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:12.891719 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7e176-0f31-4703-8336-51ccb1e4c529 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:12.891996 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7e176-0f31-4703-8336-51ccb1e4c529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:12.911905 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f7e176-0f31-4703-8336-51ccb1e4c529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:12.913460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f7e176-0f31-4703-8336-51ccb1e4c529 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:12.937384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 858/1703] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:13.154345 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606f13f-5241-4e8e-b874-daa1855637bd None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:13.155843 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6606f13f-5241-4e8e-b874-daa1855637bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:13.202858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606f13f-5241-4e8e-b874-daa1855637bd None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:13.218824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151ed81-d0b4-4370-869b-e6ccbfee644c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:13.219478 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151ed81-d0b4-4370-869b-e6ccbfee644c service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:13.219764 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151ed81-d0b4-4370-869b-e6ccbfee644c service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:13.220045 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151ed81-d0b4-4370-869b-e6ccbfee644c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:13.259123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b151ed81-d0b4-4370-869b-e6ccbfee644c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:13.261046 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b151ed81-d0b4-4370-869b-e6ccbfee644c service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:13.312644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 846/1704] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:13.497182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670d119-efcc-4783-9d54-23ac1c37d4b9 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:13.498328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670d119-efcc-4783-9d54-23ac1c37d4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:13.498876 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670d119-efcc-4783-9d54-23ac1c37d4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:13.573273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-b670d119-efcc-4783-9d54-23ac1c37d4b9 None None] MFA Rules not processed for user `22bb7775bae64374b11462a53853e54f`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:13.578830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b670d119-efcc-4783-9d54-23ac1c37d4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:13.765221 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 859/1705] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:14.050910 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151ed81-d0b4-4370-869b-e6ccbfee644c service nova] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:14.052605 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b151ed81-d0b4-4370-869b-e6ccbfee644c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:14.188202 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa70ed9-4227-43b5-9333-53c348286d3b None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:14.189110 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa70ed9-4227-43b5-9333-53c348286d3b None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:14.189727 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa70ed9-4227-43b5-9333-53c348286d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:14.223863 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-6aa70ed9-4227-43b5-9333-53c348286d3b None None] MFA Rules not processed for user `140059c5b30c426ab6d7f1236eb5f684`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:14.225346 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-6aa70ed9-4227-43b5-9333-53c348286d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:14.250025 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151ed81-d0b4-4370-869b-e6ccbfee644c service nova] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:14.258760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ece434-0f23-46aa-b19a-8fea225acb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:14.260559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ece434-0f23-46aa-b19a-8fea225acb28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:14.261235 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ece434-0f23-46aa-b19a-8fea225acb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:14.262739 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ece434-0f23-46aa-b19a-8fea225acb28 admin admin] PATH_INFO: `/v3/projects/033aea18564f471faaa2c9b9672eaaeb` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:14.264954 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ece434-0f23-46aa-b19a-8fea225acb28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=033aea18564f471faaa2c9b9672eaaeb)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:14.269270 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ece434-0f23-46aa-b19a-8fea225acb28 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:14.332099 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:14.357031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-c8ece434-0f23-46aa-b19a-8fea225acb28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '033aea18564f471faaa2c9b9672eaaeb', 'request_id': 'req-c8ece434-0f23-46aa-b19a-8fea225acb28'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:14.367156 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 860/1706] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:14.375684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-c8ece434-0f23-46aa-b19a-8fea225acb28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '033aea18564f471faaa2c9b9672eaaeb', 'request_id': 'req-c8ece434-0f23-46aa-b19a-8fea225acb28'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:14.389996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-c8ece434-0f23-46aa-b19a-8fea225acb28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '033aea18564f471faaa2c9b9672eaaeb', 'request_id': 'req-c8ece434-0f23-46aa-b19a-8fea225acb28'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:14.407485 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-c8ece434-0f23-46aa-b19a-8fea225acb28 admin admin] The token cache is being invalidate because project 033aea18564f471faaa2c9b9672eaaeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:14.408234 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-c8ece434-0f23-46aa-b19a-8fea225acb28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:14.409352 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-c8ece434-0f23-46aa-b19a-8fea225acb28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:14.411343 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 847/1707] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:14 2022] DELETE /identity/v3/projects/033aea18564f471faaa2c9b9672eaaeb => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:14.416517 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e78aa-710f-46cb-b013-32553413210b None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:14.416816 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e78aa-710f-46cb-b013-32553413210b None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:14.420665 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e78aa-710f-46cb-b013-32553413210b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:14.472976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-b84e78aa-710f-46cb-b013-32553413210b None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:14.474256 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b84e78aa-710f-46cb-b013-32553413210b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:14.649174 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 861/1708] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:14.651828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac87f55-8f32-4219-9cab-dd6de7243943 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:14.652122 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac87f55-8f32-4219-9cab-dd6de7243943 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:14.652400 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac87f55-8f32-4219-9cab-dd6de7243943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:14.718372 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-4ac87f55-8f32-4219-9cab-dd6de7243943 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:14.719862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-4ac87f55-8f32-4219-9cab-dd6de7243943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:14.745265 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 848/1709] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:36:14.749795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e78aa-710f-46cb-b013-32553413210b None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:14.766314 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e78aa-710f-46cb-b013-32553413210b None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:14.772415 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bd254-fd4e-4660-a53e-40cc689c69b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:14.773494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bd254-fd4e-4660-a53e-40cc689c69b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:14.774100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bd254-fd4e-4660-a53e-40cc689c69b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:14.774654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bd254-fd4e-4660-a53e-40cc689c69b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:14.776274 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65bd254-fd4e-4660-a53e-40cc689c69b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:36:14.776824 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65bd254-fd4e-4660-a53e-40cc689c69b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:14.778766 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65bd254-fd4e-4660-a53e-40cc689c69b0 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:14.806665 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 862/1710] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:36:14.812227 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac87f55-8f32-4219-9cab-dd6de7243943 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:14.825040 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac87f55-8f32-4219-9cab-dd6de7243943 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:14.834435 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9ad70-2e3a-464e-b37c-7b430d01a738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:14.835837 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9ad70-2e3a-464e-b37c-7b430d01a738 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:14.836544 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9ad70-2e3a-464e-b37c-7b430d01a738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:14.837103 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9ad70-2e3a-464e-b37c-7b430d01a738 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:14.838379 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9ad70-2e3a-464e-b37c-7b430d01a738 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:14.840758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9ad70-2e3a-464e-b37c-7b430d01a738 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:14.889984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 849/1711] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:14 2022] POST /identity/v3/projects => generated 379 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:14.894790 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bd254-fd4e-4660-a53e-40cc689c69b0 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:14.910987 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bd254-fd4e-4660-a53e-40cc689c69b0 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:15.057223 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e804c3b-ed3d-4263-a711-e7214bc2a182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:15.058081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e804c3b-ed3d-4263-a711-e7214bc2a182 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:15.058247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e804c3b-ed3d-4263-a711-e7214bc2a182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:15.058454 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e804c3b-ed3d-4263-a711-e7214bc2a182 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:15.059096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e804c3b-ed3d-4263-a711-e7214bc2a182 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:15.071519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e804c3b-ed3d-4263-a711-e7214bc2a182 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:15.194372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 863/1712] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:14 2022] POST /identity/v3/users => generated 290 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:15.197031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9ad70-2e3a-464e-b37c-7b430d01a738 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:15.213612 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9ad70-2e3a-464e-b37c-7b430d01a738 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:15.249689 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938f886-e36d-4106-ad9a-44c94695424f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:15.251063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938f886-e36d-4106-ad9a-44c94695424f admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:15.251988 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938f886-e36d-4106-ad9a-44c94695424f admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:15.252683 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938f886-e36d-4106-ad9a-44c94695424f admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:15.253941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938f886-e36d-4106-ad9a-44c94695424f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:15.257244 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938f886-e36d-4106-ad9a-44c94695424f admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:15.288741 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 850/1713] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:36:15.293154 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e804c3b-ed3d-4263-a711-e7214bc2a182 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:15.311423 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e804c3b-ed3d-4263-a711-e7214bc2a182 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:15.330220 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01deed3-c183-4add-9553-77b9d71f214b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:15.333691 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01deed3-c183-4add-9553-77b9d71f214b admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:15.333944 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01deed3-c183-4add-9553-77b9d71f214b admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:15.334224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01deed3-c183-4add-9553-77b9d71f214b admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:15.339269 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01deed3-c183-4add-9553-77b9d71f214b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:15.344085 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01deed3-c183-4add-9553-77b9d71f214b admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:15.389665 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 864/1714] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:15.395395 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938f886-e36d-4106-ad9a-44c94695424f admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:15.434492 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938f886-e36d-4106-ad9a-44c94695424f admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:15.454961 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f411716-576e-4cf9-a8d4-3fb2a5048f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:15.456188 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f411716-576e-4cf9-a8d4-3fb2a5048f93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:15.456462 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f411716-576e-4cf9-a8d4-3fb2a5048f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:15.457129 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f411716-576e-4cf9-a8d4-3fb2a5048f93 admin admin] PATH_INFO: `/v3/projects/deaeada46274450fb6816e2e5c7f65a0/users/b564242296ed48aea4bf163f40e3643e/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:15.495696 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f411716-576e-4cf9-a8d4-3fb2a5048f93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=deaeada46274450fb6816e2e5c7f65a0, user_id=b564242296ed48aea4bf163f40e3643e, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:15.500639 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f411716-576e-4cf9-a8d4-3fb2a5048f93 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:15.528134 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 851/1715] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:15 2022] PUT /identity/v3/projects/deaeada46274450fb6816e2e5c7f65a0/users/b564242296ed48aea4bf163f40e3643e/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:15.538460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43d165-b395-40b1-8552-99f94835c74b None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:15.538749 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43d165-b395-40b1-8552-99f94835c74b None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:15.539023 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43d165-b395-40b1-8552-99f94835c74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:15.573518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-ec43d165-b395-40b1-8552-99f94835c74b None None] MFA Rules not processed for user `b564242296ed48aea4bf163f40e3643e`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:15.574656 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-ec43d165-b395-40b1-8552-99f94835c74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:15.695081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:15.763286 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 865/1716] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:15.768210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f411716-576e-4cf9-a8d4-3fb2a5048f93 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:15.775675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:15.958975 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f411716-576e-4cf9-a8d4-3fb2a5048f93 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:15.967553 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92908a-9e49-4b8d-bc91-ef6988aa7fcf None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:15.968423 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92908a-9e49-4b8d-bc91-ef6988aa7fcf None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:15.969328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92908a-9e49-4b8d-bc91-ef6988aa7fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:15.981661 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b3c6cc-b77f-4695-ad00-54e1310ce477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:15.982397 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b3c6cc-b77f-4695-ad00-54e1310ce477 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:15.982686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b3c6cc-b77f-4695-ad00-54e1310ce477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:15.982964 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b3c6cc-b77f-4695-ad00-54e1310ce477 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:15.987901 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b3c6cc-b77f-4695-ad00-54e1310ce477 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:15.997704 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b3c6cc-b77f-4695-ad00-54e1310ce477 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:16.102038 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-4b92908a-9e49-4b8d-bc91-ef6988aa7fcf None None] MFA Rules not processed for user `1c0838069cba4d1389ec6fc809db70a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:16.103272 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-4b92908a-9e49-4b8d-bc91-ef6988aa7fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:16.156862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 852/1717] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:15 2022] POST /identity/v3/projects => generated 379 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:36:16.161818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b3c6cc-b77f-4695-ad00-54e1310ce477 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:16.339003 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 866/1718] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:36:16.459199 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b3c6cc-b77f-4695-ad00-54e1310ce477 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:16.486252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d0ffd-edc9-47aa-9fc6-2df059fead59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:16.486909 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d0ffd-edc9-47aa-9fc6-2df059fead59 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:16.487198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d0ffd-edc9-47aa-9fc6-2df059fead59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:16.487496 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d0ffd-edc9-47aa-9fc6-2df059fead59 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:16.488169 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d0ffd-edc9-47aa-9fc6-2df059fead59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:16.495165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d0ffd-edc9-47aa-9fc6-2df059fead59 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:16.573883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92908a-9e49-4b8d-bc91-ef6988aa7fcf None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:16.575839 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 853/1719] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:16 2022] POST /identity/v3/users => generated 296 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:16.595098 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92908a-9e49-4b8d-bc91-ef6988aa7fcf None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:16.611585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a58e5-b4fd-44d8-aa66-3d1ebaa569f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:16.614252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a58e5-b4fd-44d8-aa66-3d1ebaa569f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:16.614734 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a58e5-b4fd-44d8-aa66-3d1ebaa569f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:16.615668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a58e5-b4fd-44d8-aa66-3d1ebaa569f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:16.619203 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a58e5-b4fd-44d8-aa66-3d1ebaa569f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:16.621505 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a58e5-b4fd-44d8-aa66-3d1ebaa569f6 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:16.673354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 867/1720] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:36:16.674752 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d0ffd-edc9-47aa-9fc6-2df059fead59 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:16.694298 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d0ffd-edc9-47aa-9fc6-2df059fead59 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:16.708277 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cedab7-30a0-4d14-8219-e84c00d416b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:16.709261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cedab7-30a0-4d14-8219-e84c00d416b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:16.709769 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cedab7-30a0-4d14-8219-e84c00d416b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:16.710252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cedab7-30a0-4d14-8219-e84c00d416b0 admin admin] PATH_INFO: `/v3/projects/8e6e1de0e8f34f98a42117717d88a27c/users/eb7134691658487ab7b7dbd80cd7c40b/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:16.763330 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cedab7-30a0-4d14-8219-e84c00d416b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e6e1de0e8f34f98a42117717d88a27c, user_id=eb7134691658487ab7b7dbd80cd7c40b, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:16.767706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cedab7-30a0-4d14-8219-e84c00d416b0 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:16.802107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 854/1721] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:16 2022] PUT /identity/v3/projects/8e6e1de0e8f34f98a42117717d88a27c/users/eb7134691658487ab7b7dbd80cd7c40b/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:16.806127 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a58e5-b4fd-44d8-aa66-3d1ebaa569f6 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:16.831951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a58e5-b4fd-44d8-aa66-3d1ebaa569f6 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:16.955223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f8fd9-a861-404f-92f2-076228ca82bb None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:16.955851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f8fd9-a861-404f-92f2-076228ca82bb None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:16.957766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f8fd9-a861-404f-92f2-076228ca82bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:16.985682 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2d6db-b368-414d-85e9-3aecf9a594c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:16.986654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2d6db-b368-414d-85e9-3aecf9a594c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:16.987145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2d6db-b368-414d-85e9-3aecf9a594c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:16.987586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2d6db-b368-414d-85e9-3aecf9a594c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:16.988619 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c2d6db-b368-414d-85e9-3aecf9a594c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:16.993180 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c2d6db-b368-414d-85e9-3aecf9a594c3 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:17.003649 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-934f8fd9-a861-404f-92f2-076228ca82bb None None] MFA Rules not processed for user `7a0bd78165c94d9e9750fe4af4f7d375`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:17.005873 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-934f8fd9-a861-404f-92f2-076228ca82bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:17.017162 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 868/1722] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:36:17.030189 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2d6db-b368-414d-85e9-3aecf9a594c3 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:17.050253 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2d6db-b368-414d-85e9-3aecf9a594c3 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:17.055983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edd6e0-6003-495f-8ffd-56454140a0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:17.056671 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edd6e0-6003-495f-8ffd-56454140a0e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:17.057053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edd6e0-6003-495f-8ffd-56454140a0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:17.057343 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edd6e0-6003-495f-8ffd-56454140a0e8 admin admin] PATH_INFO: `/v3/projects/8e6e1de0e8f34f98a42117717d88a27c/users/eb7134691658487ab7b7dbd80cd7c40b/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:17.059507 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edd6e0-6003-495f-8ffd-56454140a0e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e6e1de0e8f34f98a42117717d88a27c, user_id=eb7134691658487ab7b7dbd80cd7c40b, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:17.063243 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edd6e0-6003-495f-8ffd-56454140a0e8 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:17.088201 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 869/1723] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:17 2022] PUT /identity/v3/projects/8e6e1de0e8f34f98a42117717d88a27c/users/eb7134691658487ab7b7dbd80cd7c40b/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:17.089993 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76cfa3-8afa-4127-a358-3fef617a8ea3 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:17.090288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76cfa3-8afa-4127-a358-3fef617a8ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:17.090570 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76cfa3-8afa-4127-a358-3fef617a8ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:17.158595 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-4c76cfa3-8afa-4127-a358-3fef617a8ea3 None None] MFA Rules not processed for user `eb7134691658487ab7b7dbd80cd7c40b`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:17.159773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-4c76cfa3-8afa-4127-a358-3fef617a8ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:17.188331 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:17.208879 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 855/1724] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:17.309535 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:17.341176 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 870/1725] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:17 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:17.353662 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f8fd9-a861-404f-92f2-076228ca82bb None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:17.355600 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-934f8fd9-a861-404f-92f2-076228ca82bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:17.459107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f8fd9-a861-404f-92f2-076228ca82bb None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:17.528437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328d165-7b95-43c9-a4dd-cfec01e290e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:17.529742 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328d165-7b95-43c9-a4dd-cfec01e290e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:17.529989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328d165-7b95-43c9-a4dd-cfec01e290e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:17.530272 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328d165-7b95-43c9-a4dd-cfec01e290e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:17.540769 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4328d165-7b95-43c9-a4dd-cfec01e290e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:17.542452 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4328d165-7b95-43c9-a4dd-cfec01e290e1 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:17.553712 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 856/1726] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:17 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:18.096425 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76cfa3-8afa-4127-a358-3fef617a8ea3 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:18.111000 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76cfa3-8afa-4127-a358-3fef617a8ea3 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:18.130136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960496f-5d54-4eac-ae4c-1bcc181e9b5a tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7134691658487ab7b7dbd80cd7c40b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e6e1de0e8f34f98a42117717d88a27c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:18.130858 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960496f-5d54-4eac-ae4c-1bcc181e9b5a tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:18.131249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960496f-5d54-4eac-ae4c-1bcc181e9b5a tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:18.131637 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960496f-5d54-4eac-ae4c-1bcc181e9b5a tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:18.133087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6960496f-5d54-4eac-ae4c-1bcc181e9b5a tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:36:18.133647 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6960496f-5d54-4eac-ae4c-1bcc181e9b5a tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:18.136400 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6960496f-5d54-4eac-ae4c-1bcc181e9b5a tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:18.167150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 871/1727] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:36:18.170339 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328d165-7b95-43c9-a4dd-cfec01e290e1 service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:18.207315 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328d165-7b95-43c9-a4dd-cfec01e290e1 service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:18.223279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5941b-0bdc-44dc-aa94-05daf43d2b6b tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7134691658487ab7b7dbd80cd7c40b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e6e1de0e8f34f98a42117717d88a27c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:18.225080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5941b-0bdc-44dc-aa94-05daf43d2b6b tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:18.225632 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5941b-0bdc-44dc-aa94-05daf43d2b6b tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:18.227892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5941b-0bdc-44dc-aa94-05daf43d2b6b tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:18.229009 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5941b-0bdc-44dc-aa94-05daf43d2b6b tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:18.234634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5941b-0bdc-44dc-aa94-05daf43d2b6b tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:18.390326 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960496f-5d54-4eac-ae4c-1bcc181e9b5a tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:18.405406 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 857/1728] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:18 2022] POST /identity/v3/projects => generated 354 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:36:18.417770 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960496f-5d54-4eac-ae4c-1bcc181e9b5a tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:18.629342 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c538a3-8fea-4e87-9ac9-f3896e6ab4c5 tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7134691658487ab7b7dbd80cd7c40b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e6e1de0e8f34f98a42117717d88a27c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:18.634802 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c538a3-8fea-4e87-9ac9-f3896e6ab4c5 tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:18.635129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c538a3-8fea-4e87-9ac9-f3896e6ab4c5 tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:18.635375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c538a3-8fea-4e87-9ac9-f3896e6ab4c5 tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:18.639872 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c538a3-8fea-4e87-9ac9-f3896e6ab4c5 tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:36:18.640388 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c538a3-8fea-4e87-9ac9-f3896e6ab4c5 tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:18.644351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c538a3-8fea-4e87-9ac9-f3896e6ab4c5 tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:18.685546 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 872/1729] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:36:18.809114 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5941b-0bdc-44dc-aa94-05daf43d2b6b tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:18.832385 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5941b-0bdc-44dc-aa94-05daf43d2b6b tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:18.845522 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60859320-7c98-4f6a-b0c9-66c2cc816e51 tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb7134691658487ab7b7dbd80cd7c40b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e6e1de0e8f34f98a42117717d88a27c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:18.846532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60859320-7c98-4f6a-b0c9-66c2cc816e51 tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:18.847004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60859320-7c98-4f6a-b0c9-66c2cc816e51 tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:18.847488 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60859320-7c98-4f6a-b0c9-66c2cc816e51 tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] PATH_INFO: `/v3/projects/d4b477a0702e4bcf820bf46e14f93bf9` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:18.850848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60859320-7c98-4f6a-b0c9-66c2cc816e51 tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d4b477a0702e4bcf820bf46e14f93bf9)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:18.853094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60859320-7c98-4f6a-b0c9-66c2cc816e51 tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:18.960405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-60859320-7c98-4f6a-b0c9-66c2cc816e51 tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4b477a0702e4bcf820bf46e14f93bf9', 'request_id': 'req-60859320-7c98-4f6a-b0c9-66c2cc816e51'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:18.970738 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-60859320-7c98-4f6a-b0c9-66c2cc816e51 tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4b477a0702e4bcf820bf46e14f93bf9', 'request_id': 'req-60859320-7c98-4f6a-b0c9-66c2cc816e51'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:18.984469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-60859320-7c98-4f6a-b0c9-66c2cc816e51 tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4b477a0702e4bcf820bf46e14f93bf9', 'request_id': 'req-60859320-7c98-4f6a-b0c9-66c2cc816e51'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:19.009714 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-60859320-7c98-4f6a-b0c9-66c2cc816e51 tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] The token cache is being invalidate because project d4b477a0702e4bcf820bf46e14f93bf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:19.010031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-60859320-7c98-4f6a-b0c9-66c2cc816e51 tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:19.010821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-60859320-7c98-4f6a-b0c9-66c2cc816e51 tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:19.019699 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 858/1730] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:18 2022] DELETE /identity/v3/projects/d4b477a0702e4bcf820bf46e14f93bf9 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:19.426598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c538a3-8fea-4e87-9ac9-f3896e6ab4c5 tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:19.429512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-33c538a3-8fea-4e87-9ac9-f3896e6ab4c5 tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:19.721230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c538a3-8fea-4e87-9ac9-f3896e6ab4c5 tempest-VolumeQuotaClassesTest-862356740 tempest-VolumeQuotaClassesTest-862356740-project-admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:19.964685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bbe40-60a2-4ce0-8cf5-1618fb885479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:19.981317 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bbe40-60a2-4ce0-8cf5-1618fb885479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:19.981724 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bbe40-60a2-4ce0-8cf5-1618fb885479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:19.981963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bbe40-60a2-4ce0-8cf5-1618fb885479 admin admin] PATH_INFO: `/v3/users/b564242296ed48aea4bf163f40e3643e` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:19.994047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652bbe40-60a2-4ce0-8cf5-1618fb885479 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b564242296ed48aea4bf163f40e3643e)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:19.996251 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652bbe40-60a2-4ce0-8cf5-1618fb885479 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:20.129941 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:20.225764 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-652bbe40-60a2-4ce0-8cf5-1618fb885479 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b564242296ed48aea4bf163f40e3643e'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:20.252938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-652bbe40-60a2-4ce0-8cf5-1618fb885479 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b564242296ed48aea4bf163f40e3643e'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:20.288629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-652bbe40-60a2-4ce0-8cf5-1618fb885479 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b564242296ed48aea4bf163f40e3643e'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:20.330202 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-652bbe40-60a2-4ce0-8cf5-1618fb885479 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b564242296ed48aea4bf163f40e3643e'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:20.332720 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-652bbe40-60a2-4ce0-8cf5-1618fb885479 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b564242296ed48aea4bf163f40e3643e'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:20.354144 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 873/1731] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:19 2022] DELETE /identity/v3/users/b564242296ed48aea4bf163f40e3643e => generated 0 bytes in 925 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:20.356650 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132e0be-29a0-48cd-9cf5-98db80aefc74 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:20.358417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132e0be-29a0-48cd-9cf5-98db80aefc74 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:20.358895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132e0be-29a0-48cd-9cf5-98db80aefc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:20.424513 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-6132e0be-29a0-48cd-9cf5-98db80aefc74 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:20.426167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6132e0be-29a0-48cd-9cf5-98db80aefc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:20.459243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:20.586024 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 859/1732] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:20.597318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bbe40-60a2-4ce0-8cf5-1618fb885479 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:20.598976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-652bbe40-60a2-4ce0-8cf5-1618fb885479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:20.875108 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bbe40-60a2-4ce0-8cf5-1618fb885479 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:20.880770 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7005b-cdd2-4efd-9892-b0f6b436b804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:20.882161 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7005b-cdd2-4efd-9892-b0f6b436b804 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:20.882647 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7005b-cdd2-4efd-9892-b0f6b436b804 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:20.883132 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7005b-cdd2-4efd-9892-b0f6b436b804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:20.900262 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7005b-cdd2-4efd-9892-b0f6b436b804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:20.902733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7005b-cdd2-4efd-9892-b0f6b436b804 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:20.936818 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 874/1733] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:21.632160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132e0be-29a0-48cd-9cf5-98db80aefc74 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:21.633854 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6132e0be-29a0-48cd-9cf5-98db80aefc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:21.662946 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132e0be-29a0-48cd-9cf5-98db80aefc74 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:21.672178 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8701a5-97ee-4137-bcbd-f4b20b6eac4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:21.673518 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8701a5-97ee-4137-bcbd-f4b20b6eac4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:21.674590 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8701a5-97ee-4137-bcbd-f4b20b6eac4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:21.675045 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8701a5-97ee-4137-bcbd-f4b20b6eac4c admin admin] PATH_INFO: `/v3/projects/deaeada46274450fb6816e2e5c7f65a0` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:21.683433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8701a5-97ee-4137-bcbd-f4b20b6eac4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=deaeada46274450fb6816e2e5c7f65a0)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:21.692642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8701a5-97ee-4137-bcbd-f4b20b6eac4c admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:21.798785 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-ac8701a5-97ee-4137-bcbd-f4b20b6eac4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deaeada46274450fb6816e2e5c7f65a0', 'request_id': 'req-ac8701a5-97ee-4137-bcbd-f4b20b6eac4c'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:21.841667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-ac8701a5-97ee-4137-bcbd-f4b20b6eac4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deaeada46274450fb6816e2e5c7f65a0', 'request_id': 'req-ac8701a5-97ee-4137-bcbd-f4b20b6eac4c'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:21.854044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-ac8701a5-97ee-4137-bcbd-f4b20b6eac4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deaeada46274450fb6816e2e5c7f65a0', 'request_id': 'req-ac8701a5-97ee-4137-bcbd-f4b20b6eac4c'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:21.878556 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-ac8701a5-97ee-4137-bcbd-f4b20b6eac4c admin admin] The token cache is being invalidate because project deaeada46274450fb6816e2e5c7f65a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:21.878772 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-ac8701a5-97ee-4137-bcbd-f4b20b6eac4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:21.882685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-ac8701a5-97ee-4137-bcbd-f4b20b6eac4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:21.900218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 860/1734] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:21 2022] DELETE /identity/v3/projects/deaeada46274450fb6816e2e5c7f65a0 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:21.904247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7005b-cdd2-4efd-9892-b0f6b436b804 service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:21.916062 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7005b-cdd2-4efd-9892-b0f6b436b804 service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:22.097899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72baac2d-d4bf-4e6e-a24b-e4506ae5fd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:22.098754 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baac2d-d4bf-4e6e-a24b-e4506ae5fd3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:22.099044 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baac2d-d4bf-4e6e-a24b-e4506ae5fd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:22.099326 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baac2d-d4bf-4e6e-a24b-e4506ae5fd3c admin admin] PATH_INFO: `/v3/users/eb7134691658487ab7b7dbd80cd7c40b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:22.100701 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72baac2d-d4bf-4e6e-a24b-e4506ae5fd3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb7134691658487ab7b7dbd80cd7c40b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:22.102925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72baac2d-d4bf-4e6e-a24b-e4506ae5fd3c admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:22.317709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:22.332748 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-72baac2d-d4bf-4e6e-a24b-e4506ae5fd3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb7134691658487ab7b7dbd80cd7c40b'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:22.362075 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-72baac2d-d4bf-4e6e-a24b-e4506ae5fd3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb7134691658487ab7b7dbd80cd7c40b'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:22.408919 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-72baac2d-d4bf-4e6e-a24b-e4506ae5fd3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb7134691658487ab7b7dbd80cd7c40b'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:22.451552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-72baac2d-d4bf-4e6e-a24b-e4506ae5fd3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb7134691658487ab7b7dbd80cd7c40b'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:22.455535 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-72baac2d-d4bf-4e6e-a24b-e4506ae5fd3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb7134691658487ab7b7dbd80cd7c40b'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:22.488533 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 875/1735] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:21 2022] DELETE /identity/v3/users/eb7134691658487ab7b7dbd80cd7c40b => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:23.185455 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8701a5-97ee-4137-bcbd-f4b20b6eac4c admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:23.187066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ac8701a5-97ee-4137-bcbd-f4b20b6eac4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:23.203450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8701a5-97ee-4137-bcbd-f4b20b6eac4c admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:23.273856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d7dea-df4f-4502-8b07-e67d99f9759b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:23.274721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d7dea-df4f-4502-8b07-e67d99f9759b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:23.275350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d7dea-df4f-4502-8b07-e67d99f9759b admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:23.275794 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d7dea-df4f-4502-8b07-e67d99f9759b admin admin] PATH_INFO: `/v3/projects/8e6e1de0e8f34f98a42117717d88a27c` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:23.277185 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d7dea-df4f-4502-8b07-e67d99f9759b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e6e1de0e8f34f98a42117717d88a27c)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:23.279960 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d7dea-df4f-4502-8b07-e67d99f9759b admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:23.331708 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:23.336340 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-2d0d7dea-df4f-4502-8b07-e67d99f9759b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e6e1de0e8f34f98a42117717d88a27c', 'request_id': 'req-2d0d7dea-df4f-4502-8b07-e67d99f9759b'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:23.345502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-2d0d7dea-df4f-4502-8b07-e67d99f9759b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e6e1de0e8f34f98a42117717d88a27c', 'request_id': 'req-2d0d7dea-df4f-4502-8b07-e67d99f9759b'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:23.355136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-2d0d7dea-df4f-4502-8b07-e67d99f9759b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e6e1de0e8f34f98a42117717d88a27c', 'request_id': 'req-2d0d7dea-df4f-4502-8b07-e67d99f9759b'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:23.368934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-2d0d7dea-df4f-4502-8b07-e67d99f9759b admin admin] The token cache is being invalidate because project 8e6e1de0e8f34f98a42117717d88a27c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:23.369123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-2d0d7dea-df4f-4502-8b07-e67d99f9759b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:23.369783 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-2d0d7dea-df4f-4502-8b07-e67d99f9759b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:23.371563 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 861/1736] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:23 2022] DELETE /identity/v3/projects/8e6e1de0e8f34f98a42117717d88a27c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:23.376177 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9678995-2eb0-4489-9032-2d9b5882d015 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:23.376708 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9678995-2eb0-4489-9032-2d9b5882d015 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:23.376791 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9678995-2eb0-4489-9032-2d9b5882d015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:23.397165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-b9678995-2eb0-4489-9032-2d9b5882d015 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:23.398358 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b9678995-2eb0-4489-9032-2d9b5882d015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:23.485981 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 876/1737] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:23.493814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425a1d3-ba4d-479f-814f-f0ba304fdd0a None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:23.494108 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425a1d3-ba4d-479f-814f-f0ba304fdd0a None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:23.494565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425a1d3-ba4d-479f-814f-f0ba304fdd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:23.521085 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-3425a1d3-ba4d-479f-814f-f0ba304fdd0a None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:23.522979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-3425a1d3-ba4d-479f-814f-f0ba304fdd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:23.533198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 862/1738] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:23.537707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9678995-2eb0-4489-9032-2d9b5882d015 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:23.566159 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9678995-2eb0-4489-9032-2d9b5882d015 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:23.584803 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39359c77-180b-4681-9f88-a04cf1523447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:23.585486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39359c77-180b-4681-9f88-a04cf1523447 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:23.585767 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39359c77-180b-4681-9f88-a04cf1523447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:23.586043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39359c77-180b-4681-9f88-a04cf1523447 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:23.592650 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39359c77-180b-4681-9f88-a04cf1523447 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:36:23.593390 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39359c77-180b-4681-9f88-a04cf1523447 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:23.595988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39359c77-180b-4681-9f88-a04cf1523447 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:23.630356 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 877/1739] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:36:23.634977 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425a1d3-ba4d-479f-814f-f0ba304fdd0a None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:23.648622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425a1d3-ba4d-479f-814f-f0ba304fdd0a None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:23.661891 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25b6bd-cbb0-4bb8-bce0-75aae016c6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:23.664786 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25b6bd-cbb0-4bb8-bce0-75aae016c6d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:23.665087 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25b6bd-cbb0-4bb8-bce0-75aae016c6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:23.665559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25b6bd-cbb0-4bb8-bce0-75aae016c6d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:23.666657 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25b6bd-cbb0-4bb8-bce0-75aae016c6d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:23.672178 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25b6bd-cbb0-4bb8-bce0-75aae016c6d6 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:23.733853 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 863/1740] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:23 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:23.737941 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39359c77-180b-4681-9f88-a04cf1523447 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:23.753254 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39359c77-180b-4681-9f88-a04cf1523447 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:23.848484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454588c9-23e1-4297-a190-58da5262e004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:23.849467 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454588c9-23e1-4297-a190-58da5262e004 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:23.849809 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454588c9-23e1-4297-a190-58da5262e004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:23.850089 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454588c9-23e1-4297-a190-58da5262e004 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:23.850849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454588c9-23e1-4297-a190-58da5262e004 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:23.852979 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454588c9-23e1-4297-a190-58da5262e004 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:23.887232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 878/1741] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:23 2022] POST /identity/v3/users => generated 294 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:23.890606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25b6bd-cbb0-4bb8-bce0-75aae016c6d6 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:23.902600 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25b6bd-cbb0-4bb8-bce0-75aae016c6d6 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:23.954290 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a16d6d-ca57-48cf-8a77-6763fbdc17b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:23.954944 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a16d6d-ca57-48cf-8a77-6763fbdc17b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:23.955229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a16d6d-ca57-48cf-8a77-6763fbdc17b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:23.955508 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a16d6d-ca57-48cf-8a77-6763fbdc17b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:23.960020 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a16d6d-ca57-48cf-8a77-6763fbdc17b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:23.961689 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a16d6d-ca57-48cf-8a77-6763fbdc17b4 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:23.996974 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 864/1742] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:24.000371 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454588c9-23e1-4297-a190-58da5262e004 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:24.017172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454588c9-23e1-4297-a190-58da5262e004 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:24.034159 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66bd9e9-84fd-467e-b6b6-248ff2e1f705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:24.036552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bd9e9-84fd-467e-b6b6-248ff2e1f705 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:24.040256 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bd9e9-84fd-467e-b6b6-248ff2e1f705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:24.041986 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bd9e9-84fd-467e-b6b6-248ff2e1f705 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:24.042914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bd9e9-84fd-467e-b6b6-248ff2e1f705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:24.044912 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bd9e9-84fd-467e-b6b6-248ff2e1f705 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:24.089497 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 879/1743] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:24.095304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a16d6d-ca57-48cf-8a77-6763fbdc17b4 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:24.114905 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a16d6d-ca57-48cf-8a77-6763fbdc17b4 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:24.132319 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17c0d7-a8b0-49f8-9291-e02573e6c2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:24.133054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17c0d7-a8b0-49f8-9291-e02573e6c2e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:24.133335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17c0d7-a8b0-49f8-9291-e02573e6c2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:24.133614 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17c0d7-a8b0-49f8-9291-e02573e6c2e4 admin admin] PATH_INFO: `/v3/projects/017e62434ed04940be72828ee8e66929/users/7e5cc956082d4c39ad2132086251be72/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:24.213318 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17c0d7-a8b0-49f8-9291-e02573e6c2e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=017e62434ed04940be72828ee8e66929, user_id=7e5cc956082d4c39ad2132086251be72, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:24.221512 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17c0d7-a8b0-49f8-9291-e02573e6c2e4 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:24.282843 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8ff14-7605-49ca-afee-e02439889daa None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:24.283670 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8ff14-7605-49ca-afee-e02439889daa None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:24.283670 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8ff14-7605-49ca-afee-e02439889daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:24.300743 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 865/1744] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:24 2022] PUT /identity/v3/projects/017e62434ed04940be72828ee8e66929/users/7e5cc956082d4c39ad2132086251be72/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:36:24.332772 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-8cd8ff14-7605-49ca-afee-e02439889daa None None] MFA Rules not processed for user `7e5cc956082d4c39ad2132086251be72`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:24.334326 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-8cd8ff14-7605-49ca-afee-e02439889daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:24.350946 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:24.427539 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 880/1745] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:24.432267 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17c0d7-a8b0-49f8-9291-e02573e6c2e4 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:24.517344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17c0d7-a8b0-49f8-9291-e02573e6c2e4 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:24.530122 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d2e36-4ce0-4fd7-8133-ce3175bc0a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:24.530859 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d2e36-4ce0-4fd7-8133-ce3175bc0a16 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:24.532443 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d2e36-4ce0-4fd7-8133-ce3175bc0a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:24.532874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d2e36-4ce0-4fd7-8133-ce3175bc0a16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:24.534844 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d2e36-4ce0-4fd7-8133-ce3175bc0a16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:24.545551 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d2e36-4ce0-4fd7-8133-ce3175bc0a16 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:24.601094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 866/1746] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:24 2022] POST /identity/v3/projects => generated 387 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:24.608277 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8ff14-7605-49ca-afee-e02439889daa None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:24.624580 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8ff14-7605-49ca-afee-e02439889daa None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:24.708806 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f763a5b-d8a5-49dd-8c7a-4c67f0941caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:24.709485 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f763a5b-d8a5-49dd-8c7a-4c67f0941caa admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:24.709774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f763a5b-d8a5-49dd-8c7a-4c67f0941caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:24.710109 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f763a5b-d8a5-49dd-8c7a-4c67f0941caa admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:24.710788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f763a5b-d8a5-49dd-8c7a-4c67f0941caa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:24.713169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f763a5b-d8a5-49dd-8c7a-4c67f0941caa admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:24.735782 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 881/1747] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:24 2022] POST /identity/v3/users => generated 294 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:24.740012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d2e36-4ce0-4fd7-8133-ce3175bc0a16 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:24.754422 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d2e36-4ce0-4fd7-8133-ce3175bc0a16 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:24.759955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0718c3-90ff-4eda-84d4-2d01ea4fe29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:24.760587 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0718c3-90ff-4eda-84d4-2d01ea4fe29a admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:24.760876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0718c3-90ff-4eda-84d4-2d01ea4fe29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:24.761192 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0718c3-90ff-4eda-84d4-2d01ea4fe29a admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:24.761894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0718c3-90ff-4eda-84d4-2d01ea4fe29a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:24.763552 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0718c3-90ff-4eda-84d4-2d01ea4fe29a admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:24.777700 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 867/1748] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:24.778875 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f763a5b-d8a5-49dd-8c7a-4c67f0941caa admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:24.789762 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f763a5b-d8a5-49dd-8c7a-4c67f0941caa admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:24.794505 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4a932-9686-4c1b-a9fc-19f2fd0f28f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:24.795715 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4a932-9686-4c1b-a9fc-19f2fd0f28f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:24.795996 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4a932-9686-4c1b-a9fc-19f2fd0f28f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:24.796265 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4a932-9686-4c1b-a9fc-19f2fd0f28f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:24.796994 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4a932-9686-4c1b-a9fc-19f2fd0f28f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:24.798743 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4a932-9686-4c1b-a9fc-19f2fd0f28f3 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:24.809815 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 882/1749] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:24.814401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0718c3-90ff-4eda-84d4-2d01ea4fe29a admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:24.824721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0718c3-90ff-4eda-84d4-2d01ea4fe29a admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:24.831453 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5cf0e7-138f-4f13-969f-d124403c95c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:24.832144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cf0e7-138f-4f13-969f-d124403c95c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:24.832517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cf0e7-138f-4f13-969f-d124403c95c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:24.832698 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cf0e7-138f-4f13-969f-d124403c95c2 admin admin] PATH_INFO: `/v3/projects/d48fee9d68394a208d73505a7590c186/users/1823867f9332447185a1eddbdec13663/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:24.850739 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5cf0e7-138f-4f13-969f-d124403c95c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d48fee9d68394a208d73505a7590c186, user_id=1823867f9332447185a1eddbdec13663, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:24.853678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5cf0e7-138f-4f13-969f-d124403c95c2 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:24.867711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 868/1750] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:24 2022] PUT /identity/v3/projects/d48fee9d68394a208d73505a7590c186/users/1823867f9332447185a1eddbdec13663/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:24.879667 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5e3eb-0db8-44ca-b02d-172fbffa32f0 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:24.880040 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5e3eb-0db8-44ca-b02d-172fbffa32f0 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:24.880206 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5e3eb-0db8-44ca-b02d-172fbffa32f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:24.929145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-51b5e3eb-0db8-44ca-b02d-172fbffa32f0 None None] MFA Rules not processed for user `1823867f9332447185a1eddbdec13663`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:24.930571 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-51b5e3eb-0db8-44ca-b02d-172fbffa32f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:25.007017 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 883/1751] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:25.011503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5cf0e7-138f-4f13-969f-d124403c95c2 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:25.032803 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:25.079684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5cf0e7-138f-4f13-969f-d124403c95c2 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:25.084677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6788b10-1d73-4f6d-8fe5-f66b58db8ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:25.085346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6788b10-1d73-4f6d-8fe5-f66b58db8ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:25.085627 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6788b10-1d73-4f6d-8fe5-f66b58db8ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:25.085899 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6788b10-1d73-4f6d-8fe5-f66b58db8ab3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:25.086569 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6788b10-1d73-4f6d-8fe5-f66b58db8ab3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:25.088724 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6788b10-1d73-4f6d-8fe5-f66b58db8ab3 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:25.121740 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 869/1752] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:25 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:36:25.123444 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b5e3eb-0db8-44ca-b02d-172fbffa32f0 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:25.134348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b5e3eb-0db8-44ca-b02d-172fbffa32f0 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:25.153745 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:25.196838 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e8cb5-e513-40fe-8be6-333b65ed97e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:25.197488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e8cb5-e513-40fe-8be6-333b65ed97e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:25.197769 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e8cb5-e513-40fe-8be6-333b65ed97e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:25.198044 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e8cb5-e513-40fe-8be6-333b65ed97e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:25.198717 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e8cb5-e513-40fe-8be6-333b65ed97e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:25.202003 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e8cb5-e513-40fe-8be6-333b65ed97e0 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:25.224933 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 884/1753] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:25 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:25.228936 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6788b10-1d73-4f6d-8fe5-f66b58db8ab3 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:25.241318 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6788b10-1d73-4f6d-8fe5-f66b58db8ab3 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:25.246267 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec758111-dfef-46e3-9a63-c136663be8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:25.246898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec758111-dfef-46e3-9a63-c136663be8ec admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:25.247288 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec758111-dfef-46e3-9a63-c136663be8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:25.247691 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec758111-dfef-46e3-9a63-c136663be8ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:25.248903 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec758111-dfef-46e3-9a63-c136663be8ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:25.251050 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec758111-dfef-46e3-9a63-c136663be8ec admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:25.262413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 870/1754] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:25.267146 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e8cb5-e513-40fe-8be6-333b65ed97e0 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:25.277796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e8cb5-e513-40fe-8be6-333b65ed97e0 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:25.284273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5479c-6e7f-4780-84e4-7fd94e0f8e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:25.284475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5479c-6e7f-4780-84e4-7fd94e0f8e90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:25.284726 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5479c-6e7f-4780-84e4-7fd94e0f8e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:25.284975 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5479c-6e7f-4780-84e4-7fd94e0f8e90 admin admin] PATH_INFO: `/v3/projects/9c3dc65274504d0888c83d256f29c52c/users/ffa78cd64fde4e12baeaadb9debcd808/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:25.319505 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd5479c-6e7f-4780-84e4-7fd94e0f8e90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c3dc65274504d0888c83d256f29c52c, user_id=ffa78cd64fde4e12baeaadb9debcd808, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:25.321754 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd5479c-6e7f-4780-84e4-7fd94e0f8e90 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:25.362616 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 885/1755] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:25 2022] PUT /identity/v3/projects/9c3dc65274504d0888c83d256f29c52c/users/ffa78cd64fde4e12baeaadb9debcd808/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:25.366629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec758111-dfef-46e3-9a63-c136663be8ec admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:25.410003 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec758111-dfef-46e3-9a63-c136663be8ec admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:25.558038 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8040f2-520d-4ba5-a926-f6ec4937b137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:25.558890 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8040f2-520d-4ba5-a926-f6ec4937b137 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:25.559277 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8040f2-520d-4ba5-a926-f6ec4937b137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:25.559823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8040f2-520d-4ba5-a926-f6ec4937b137 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:25.560548 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8040f2-520d-4ba5-a926-f6ec4937b137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:25.564346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8040f2-520d-4ba5-a926-f6ec4937b137 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:25.582017 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 871/1756] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:36:25.583254 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5479c-6e7f-4780-84e4-7fd94e0f8e90 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:25.599769 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5479c-6e7f-4780-84e4-7fd94e0f8e90 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:25.630221 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813e4ba-6027-4269-b7dc-6c61814d4c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:25.630985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813e4ba-6027-4269-b7dc-6c61814d4c2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:25.631262 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813e4ba-6027-4269-b7dc-6c61814d4c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:25.631533 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813e4ba-6027-4269-b7dc-6c61814d4c2b admin admin] PATH_INFO: `/v3/projects/9c3dc65274504d0888c83d256f29c52c/users/ffa78cd64fde4e12baeaadb9debcd808/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:25.637374 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1813e4ba-6027-4269-b7dc-6c61814d4c2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c3dc65274504d0888c83d256f29c52c, user_id=ffa78cd64fde4e12baeaadb9debcd808, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:25.647349 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1813e4ba-6027-4269-b7dc-6c61814d4c2b admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:25.677653 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 886/1757] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:25 2022] PUT /identity/v3/projects/9c3dc65274504d0888c83d256f29c52c/users/ffa78cd64fde4e12baeaadb9debcd808/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:25.687879 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f81d24-782e-427d-8e7e-086634edf702 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:25.688243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f81d24-782e-427d-8e7e-086634edf702 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:25.688444 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f81d24-782e-427d-8e7e-086634edf702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:25.778824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-13f81d24-782e-427d-8e7e-086634edf702 None None] MFA Rules not processed for user `ffa78cd64fde4e12baeaadb9debcd808`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:25.782646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-13f81d24-782e-427d-8e7e-086634edf702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:25.919057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 872/1758] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:26.071307 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813e4ba-6027-4269-b7dc-6c61814d4c2b admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:26.079405 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-1813e4ba-6027-4269-b7dc-6c61814d4c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:26.207033 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813e4ba-6027-4269-b7dc-6c61814d4c2b admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:26.216096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e255426-ef8d-437f-b8de-f83a909b5226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:26.217044 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e255426-ef8d-437f-b8de-f83a909b5226 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:26.217426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e255426-ef8d-437f-b8de-f83a909b5226 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:26.218028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e255426-ef8d-437f-b8de-f83a909b5226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:26.237584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e255426-ef8d-437f-b8de-f83a909b5226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:26.239248 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e255426-ef8d-437f-b8de-f83a909b5226 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:26.264776 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 887/1759] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:27.201748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f81d24-782e-427d-8e7e-086634edf702 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:27.214704 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f81d24-782e-427d-8e7e-086634edf702 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:27.219576 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8f3d7-937e-45fd-8d1b-e1c099d444ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:27.220431 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8f3d7-937e-45fd-8d1b-e1c099d444ab service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:27.221104 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8f3d7-937e-45fd-8d1b-e1c099d444ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:27.221555 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8f3d7-937e-45fd-8d1b-e1c099d444ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:27.232707 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:27.324127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f8f3d7-937e-45fd-8d1b-e1c099d444ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:27.326981 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f8f3d7-937e-45fd-8d1b-e1c099d444ab service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:27.357890 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 873/1760] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:28.223043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f3e78-d8aa-41f0-912e-383c0ec08deb None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:28.223572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f3e78-d8aa-41f0-912e-383c0ec08deb None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:28.223572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f3e78-d8aa-41f0-912e-383c0ec08deb None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:28.225660 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 888/1761] 10.222.0.74 () {54 vars in 921 bytes} [Wed Jul 27 16:36:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:36:28.243625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec07dde-0e64-4b98-89b0-ba1a79a479fc None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:28.244090 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec07dde-0e64-4b98-89b0-ba1a79a479fc None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:28.244181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec07dde-0e64-4b98-89b0-ba1a79a479fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:28.280925 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-7ec07dde-0e64-4b98-89b0-ba1a79a479fc None None] MFA Rules not processed for user `56dfafd15e284b7986ba07dd3819bf65`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:28.283886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-7ec07dde-0e64-4b98-89b0-ba1a79a479fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:28.392799 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 874/1762] 10.222.0.74 () {58 vars in 1018 bytes} [Wed Jul 27 16:36:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:28.558625 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f3e78-d8aa-41f0-912e-383c0ec08deb None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:28.561229 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-532f3e78-d8aa-41f0-912e-383c0ec08deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:28.592750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f3e78-d8aa-41f0-912e-383c0ec08deb None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:28.601946 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed56a25-3608-4dbd-96a1-f9dfd025ae5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:28.602663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed56a25-3608-4dbd-96a1-f9dfd025ae5b service nova] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:28.603011 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed56a25-3608-4dbd-96a1-f9dfd025ae5b service nova] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:28.603355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed56a25-3608-4dbd-96a1-f9dfd025ae5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:28.614684 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed56a25-3608-4dbd-96a1-f9dfd025ae5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:28.617551 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed56a25-3608-4dbd-96a1-f9dfd025ae5b service nova] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:28.631745 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 889/1763] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:32.012182 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a041c-1b39-485f-81a7-ab1cedd6398f None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:32.013143 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a041c-1b39-485f-81a7-ab1cedd6398f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:32.014235 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a041c-1b39-485f-81a7-ab1cedd6398f None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:32.016062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 875/1764] 10.222.0.74 () {54 vars in 921 bytes} [Wed Jul 27 16:36:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:36:32.040150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98709944-9c3e-4866-9980-6ec55bdc32dd None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:32.041065 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98709944-9c3e-4866-9980-6ec55bdc32dd None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:32.041547 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98709944-9c3e-4866-9980-6ec55bdc32dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:32.085949 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-98709944-9c3e-4866-9980-6ec55bdc32dd None None] MFA Rules not processed for user `56dfafd15e284b7986ba07dd3819bf65`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:32.087243 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-98709944-9c3e-4866-9980-6ec55bdc32dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:32.108777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 890/1765] 10.222.0.74 () {58 vars in 1018 bytes} [Wed Jul 27 16:36:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:32.337521 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a041c-1b39-485f-81a7-ab1cedd6398f None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:32.360732 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a041c-1b39-485f-81a7-ab1cedd6398f None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:32.371247 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e818a-4922-4d35-8458-4ee91a33bab7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:32.372517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e818a-4922-4d35-8458-4ee91a33bab7 service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:32.373867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e818a-4922-4d35-8458-4ee91a33bab7 service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:32.374986 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e818a-4922-4d35-8458-4ee91a33bab7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:32.395036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e818a-4922-4d35-8458-4ee91a33bab7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:32.398231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e818a-4922-4d35-8458-4ee91a33bab7 service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:32.423580 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 876/1766] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:32.542873 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98709944-9c3e-4866-9980-6ec55bdc32dd None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:32.544470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-98709944-9c3e-4866-9980-6ec55bdc32dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:32.595307 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e818a-4922-4d35-8458-4ee91a33bab7 service nova] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:32.596786 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-4b7e818a-4922-4d35-8458-4ee91a33bab7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:32.646288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:32.655116 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98709944-9c3e-4866-9980-6ec55bdc32dd None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:32.664865 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16dcb4d-b80e-44dc-9579-0699ff7a6fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:32.666060 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16dcb4d-b80e-44dc-9579-0699ff7a6fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:32.666640 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16dcb4d-b80e-44dc-9579-0699ff7a6fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:32.667097 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16dcb4d-b80e-44dc-9579-0699ff7a6fb2 admin admin] PATH_INFO: `/v3/users/4552b96c7500492ea4dc047c6dcae3ab` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:32.668966 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16dcb4d-b80e-44dc-9579-0699ff7a6fb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4552b96c7500492ea4dc047c6dcae3ab)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:32.672510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16dcb4d-b80e-44dc-9579-0699ff7a6fb2 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:32.705261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e818a-4922-4d35-8458-4ee91a33bab7 service nova] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:32.713729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661251db-47b3-46eb-b376-31e0b189e6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:32.714921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661251db-47b3-46eb-b376-31e0b189e6e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:32.717880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661251db-47b3-46eb-b376-31e0b189e6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:32.717880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661251db-47b3-46eb-b376-31e0b189e6e7 admin admin] PATH_INFO: `/v3/users/3ada16d9d23b4e80a99df9ffc53f2185` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:32.724146 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661251db-47b3-46eb-b376-31e0b189e6e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ada16d9d23b4e80a99df9ffc53f2185)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:32.726921 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661251db-47b3-46eb-b376-31e0b189e6e7 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:32.758097 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f16dcb4d-b80e-44dc-9579-0699ff7a6fb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4552b96c7500492ea4dc047c6dcae3ab'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:32.775629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f16dcb4d-b80e-44dc-9579-0699ff7a6fb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4552b96c7500492ea4dc047c6dcae3ab'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:32.794672 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:32.801508 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f16dcb4d-b80e-44dc-9579-0699ff7a6fb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4552b96c7500492ea4dc047c6dcae3ab'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:32.829497 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-661251db-47b3-46eb-b376-31e0b189e6e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ada16d9d23b4e80a99df9ffc53f2185'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:32.847913 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-661251db-47b3-46eb-b376-31e0b189e6e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ada16d9d23b4e80a99df9ffc53f2185'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:32.848532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f16dcb4d-b80e-44dc-9579-0699ff7a6fb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4552b96c7500492ea4dc047c6dcae3ab'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:32.851075 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f16dcb4d-b80e-44dc-9579-0699ff7a6fb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4552b96c7500492ea4dc047c6dcae3ab'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:32.865488 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-661251db-47b3-46eb-b376-31e0b189e6e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ada16d9d23b4e80a99df9ffc53f2185'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:32.876109 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 891/1767] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:32 2022] DELETE /identity/v3/users/4552b96c7500492ea4dc047c6dcae3ab => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:32.879837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e8c8e-8eb1-4854-aa9b-f4ce0b8190a2 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:32.880150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e8c8e-8eb1-4854-aa9b-f4ce0b8190a2 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:32.881964 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e8c8e-8eb1-4854-aa9b-f4ce0b8190a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:32.898412 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-661251db-47b3-46eb-b376-31e0b189e6e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ada16d9d23b4e80a99df9ffc53f2185'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:32.899842 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-661251db-47b3-46eb-b376-31e0b189e6e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ada16d9d23b4e80a99df9ffc53f2185'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:32.913120 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 877/1768] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:32 2022] DELETE /identity/v3/users/3ada16d9d23b4e80a99df9ffc53f2185 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:36:32.917799 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb539c-b286-4d1c-a572-3306c5d34820 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:32.918095 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb539c-b286-4d1c-a572-3306c5d34820 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:32.918381 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb539c-b286-4d1c-a572-3306c5d34820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:32.934484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-f55e8c8e-8eb1-4854-aa9b-f4ce0b8190a2 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:32.936218 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f55e8c8e-8eb1-4854-aa9b-f4ce0b8190a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:32.945829 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-c0cb539c-b286-4d1c-a572-3306c5d34820 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:32.947349 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c0cb539c-b286-4d1c-a572-3306c5d34820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:33.034492 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 892/1769] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:33.046719 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 878/1770] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:33.047495 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e8c8e-8eb1-4854-aa9b-f4ce0b8190a2 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:33.051129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f55e8c8e-8eb1-4854-aa9b-f4ce0b8190a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:33.058619 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb539c-b286-4d1c-a572-3306c5d34820 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:33.061264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c0cb539c-b286-4d1c-a572-3306c5d34820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:33.123736 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e8c8e-8eb1-4854-aa9b-f4ce0b8190a2 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:33.131847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fdf27-40fb-4a0d-9a25-1587b338eaf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:33.132646 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fdf27-40fb-4a0d-9a25-1587b338eaf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:33.133242 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fdf27-40fb-4a0d-9a25-1587b338eaf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:33.133849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fdf27-40fb-4a0d-9a25-1587b338eaf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:33.141287 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb539c-b286-4d1c-a572-3306c5d34820 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:33.144788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9fdf27-40fb-4a0d-9a25-1587b338eaf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:33.145499 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f5365-3edb-4f12-a10b-be4c1259a7e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:33.146312 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9fdf27-40fb-4a0d-9a25-1587b338eaf8 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:33.146562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f5365-3edb-4f12-a10b-be4c1259a7e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:33.147216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f5365-3edb-4f12-a10b-be4c1259a7e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:33.147652 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f5365-3edb-4f12-a10b-be4c1259a7e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:33.158063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f5365-3edb-4f12-a10b-be4c1259a7e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:33.158239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 893/1771] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:33.160331 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f5365-3edb-4f12-a10b-be4c1259a7e5 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:33.192099 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 879/1772] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:33.259629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fdf27-40fb-4a0d-9a25-1587b338eaf8 service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:33.261077 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-6f9fdf27-40fb-4a0d-9a25-1587b338eaf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:33.287554 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:33.345228 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fdf27-40fb-4a0d-9a25-1587b338eaf8 service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:33.350316 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d1b96-35ba-4890-8be2-29879e9ae15f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:33.351053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d1b96-35ba-4890-8be2-29879e9ae15f service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:33.351341 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d1b96-35ba-4890-8be2-29879e9ae15f service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:33.351736 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d1b96-35ba-4890-8be2-29879e9ae15f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:33.353858 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-724d1b96-35ba-4890-8be2-29879e9ae15f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:33.447609 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d1b96-35ba-4890-8be2-29879e9ae15f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:33.449197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d1b96-35ba-4890-8be2-29879e9ae15f service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:33.460360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 894/1773] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:36:34.088827 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f5365-3edb-4f12-a10b-be4c1259a7e5 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:34.092802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6d6f5365-3edb-4f12-a10b-be4c1259a7e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:34.138705 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f5365-3edb-4f12-a10b-be4c1259a7e5 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:34.170512 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604e746-7801-4c52-948b-86407a53fa23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:34.171258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604e746-7801-4c52-948b-86407a53fa23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:34.171586 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604e746-7801-4c52-948b-86407a53fa23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:34.172459 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604e746-7801-4c52-948b-86407a53fa23 admin admin] PATH_INFO: `/v3/projects/62153ff309e1417eb182db62eaa5b48b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:34.182849 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604e746-7801-4c52-948b-86407a53fa23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62153ff309e1417eb182db62eaa5b48b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:34.185801 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604e746-7801-4c52-948b-86407a53fa23 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:34.351591 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d1b96-35ba-4890-8be2-29879e9ae15f service cinder] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:34.353818 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-724d1b96-35ba-4890-8be2-29879e9ae15f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:34.359203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:34.369027 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-3604e746-7801-4c52-948b-86407a53fa23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62153ff309e1417eb182db62eaa5b48b', 'request_id': 'req-3604e746-7801-4c52-948b-86407a53fa23'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:34.387940 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-3604e746-7801-4c52-948b-86407a53fa23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62153ff309e1417eb182db62eaa5b48b', 'request_id': 'req-3604e746-7801-4c52-948b-86407a53fa23'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:34.407325 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-3604e746-7801-4c52-948b-86407a53fa23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62153ff309e1417eb182db62eaa5b48b', 'request_id': 'req-3604e746-7801-4c52-948b-86407a53fa23'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:34.414101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d1b96-35ba-4890-8be2-29879e9ae15f service cinder] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:34.439386 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-3604e746-7801-4c52-948b-86407a53fa23 admin admin] The token cache is being invalidate because project 62153ff309e1417eb182db62eaa5b48b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:34.439796 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-3604e746-7801-4c52-948b-86407a53fa23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:34.441534 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-3604e746-7801-4c52-948b-86407a53fa23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:34.445795 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 880/1774] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:34 2022] DELETE /identity/v3/projects/62153ff309e1417eb182db62eaa5b48b => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:34.452346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604e746-7801-4c52-948b-86407a53fa23 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:34.454333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-3604e746-7801-4c52-948b-86407a53fa23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:34.499770 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00c991-e0dc-41eb-a8b9-5a740faa1a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:34.500566 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00c991-e0dc-41eb-a8b9-5a740faa1a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:34.500950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00c991-e0dc-41eb-a8b9-5a740faa1a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:34.501309 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00c991-e0dc-41eb-a8b9-5a740faa1a53 admin admin] PATH_INFO: `/v3/projects/27f1eace103144f197997ef13c78c7ca` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:34.503430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00c991-e0dc-41eb-a8b9-5a740faa1a53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27f1eace103144f197997ef13c78c7ca)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:34.505714 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00c991-e0dc-41eb-a8b9-5a740faa1a53 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:34.567679 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604e746-7801-4c52-948b-86407a53fa23 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:34.575213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df6fc5-f4b9-40ec-a584-b13e7597257b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:34.582054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df6fc5-f4b9-40ec-a584-b13e7597257b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:34.583069 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df6fc5-f4b9-40ec-a584-b13e7597257b admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:34.583441 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df6fc5-f4b9-40ec-a584-b13e7597257b admin admin] PATH_INFO: `/v3/users/c4ae96bd492243b4abc87b70c3a858cc` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:34.585028 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df6fc5-f4b9-40ec-a584-b13e7597257b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4ae96bd492243b4abc87b70c3a858cc)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:34.587180 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df6fc5-f4b9-40ec-a584-b13e7597257b admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:34.630238 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-eb00c991-e0dc-41eb-a8b9-5a740faa1a53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27f1eace103144f197997ef13c78c7ca', 'request_id': 'req-eb00c991-e0dc-41eb-a8b9-5a740faa1a53'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:34.668959 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-eb00c991-e0dc-41eb-a8b9-5a740faa1a53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27f1eace103144f197997ef13c78c7ca', 'request_id': 'req-eb00c991-e0dc-41eb-a8b9-5a740faa1a53'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:34.672404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-06df6fc5-f4b9-40ec-a584-b13e7597257b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4ae96bd492243b4abc87b70c3a858cc'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:34.679893 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-eb00c991-e0dc-41eb-a8b9-5a740faa1a53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27f1eace103144f197997ef13c78c7ca', 'request_id': 'req-eb00c991-e0dc-41eb-a8b9-5a740faa1a53'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:34.683292 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-06df6fc5-f4b9-40ec-a584-b13e7597257b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4ae96bd492243b4abc87b70c3a858cc'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:34.697293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-06df6fc5-f4b9-40ec-a584-b13e7597257b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4ae96bd492243b4abc87b70c3a858cc'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:34.706262 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-eb00c991-e0dc-41eb-a8b9-5a740faa1a53 admin admin] The token cache is being invalidate because project 27f1eace103144f197997ef13c78c7ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:34.706262 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-eb00c991-e0dc-41eb-a8b9-5a740faa1a53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:34.706827 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-eb00c991-e0dc-41eb-a8b9-5a740faa1a53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:34.709465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 895/1775] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:34 2022] DELETE /identity/v3/projects/27f1eace103144f197997ef13c78c7ca => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:34.714273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00c991-e0dc-41eb-a8b9-5a740faa1a53 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:34.716520 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-eb00c991-e0dc-41eb-a8b9-5a740faa1a53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:34.723538 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-06df6fc5-f4b9-40ec-a584-b13e7597257b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4ae96bd492243b4abc87b70c3a858cc'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:34.724781 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-06df6fc5-f4b9-40ec-a584-b13e7597257b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4ae96bd492243b4abc87b70c3a858cc'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:34.726371 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:34.744426 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 881/1776] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:34 2022] DELETE /identity/v3/users/c4ae96bd492243b4abc87b70c3a858cc => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:34.855231 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00c991-e0dc-41eb-a8b9-5a740faa1a53 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:34.957405 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ce4e6-4239-4b60-871e-410547b839e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:34.958129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ce4e6-4239-4b60-871e-410547b839e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:34.959445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ce4e6-4239-4b60-871e-410547b839e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:34.959883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ce4e6-4239-4b60-871e-410547b839e6 admin admin] PATH_INFO: `/v3/users/36d5144a6f254f2d99302e5b46115ad9` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:34.962171 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ce4e6-4239-4b60-871e-410547b839e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36d5144a6f254f2d99302e5b46115ad9)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:34.967050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ce4e6-4239-4b60-871e-410547b839e6 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:35.041111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a86ce4e6-4239-4b60-871e-410547b839e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36d5144a6f254f2d99302e5b46115ad9'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:35.052484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a86ce4e6-4239-4b60-871e-410547b839e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36d5144a6f254f2d99302e5b46115ad9'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:35.070166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a86ce4e6-4239-4b60-871e-410547b839e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36d5144a6f254f2d99302e5b46115ad9'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:35.078387 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a86ce4e6-4239-4b60-871e-410547b839e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36d5144a6f254f2d99302e5b46115ad9'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:35.079331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a86ce4e6-4239-4b60-871e-410547b839e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36d5144a6f254f2d99302e5b46115ad9'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:35.086274 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 896/1777] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:34 2022] DELETE /identity/v3/users/36d5144a6f254f2d99302e5b46115ad9 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:35.143403 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df6fc5-f4b9-40ec-a584-b13e7597257b admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:35.144977 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-06df6fc5-f4b9-40ec-a584-b13e7597257b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:35.179528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df6fc5-f4b9-40ec-a584-b13e7597257b admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:35.271002 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:35.287024 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ee68a-eed9-4676-8eff-dcd43515c31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:35.288141 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ee68a-eed9-4676-8eff-dcd43515c31c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:35.289827 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ee68a-eed9-4676-8eff-dcd43515c31c admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:35.290312 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ee68a-eed9-4676-8eff-dcd43515c31c admin admin] PATH_INFO: `/v3/projects/9cb1b317b9bf4c7fa19c3720ac4986d5` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:35.292289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ee68a-eed9-4676-8eff-dcd43515c31c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cb1b317b9bf4c7fa19c3720ac4986d5)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:35.296306 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ee68a-eed9-4676-8eff-dcd43515c31c admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:35.404853 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-1c8ee68a-eed9-4676-8eff-dcd43515c31c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cb1b317b9bf4c7fa19c3720ac4986d5', 'request_id': 'req-1c8ee68a-eed9-4676-8eff-dcd43515c31c'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:35.413982 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-1c8ee68a-eed9-4676-8eff-dcd43515c31c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cb1b317b9bf4c7fa19c3720ac4986d5', 'request_id': 'req-1c8ee68a-eed9-4676-8eff-dcd43515c31c'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:35.430692 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-1c8ee68a-eed9-4676-8eff-dcd43515c31c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cb1b317b9bf4c7fa19c3720ac4986d5', 'request_id': 'req-1c8ee68a-eed9-4676-8eff-dcd43515c31c'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:35.445729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-1c8ee68a-eed9-4676-8eff-dcd43515c31c admin admin] The token cache is being invalidate because project 9cb1b317b9bf4c7fa19c3720ac4986d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:35.446024 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-1c8ee68a-eed9-4676-8eff-dcd43515c31c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:35.446747 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-1c8ee68a-eed9-4676-8eff-dcd43515c31c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:35.448346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 882/1778] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:35 2022] DELETE /identity/v3/projects/9cb1b317b9bf4c7fa19c3720ac4986d5 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:35.453524 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74f2c8-3d79-4d4b-8a15-60b01dad03e2 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:35.453957 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74f2c8-3d79-4d4b-8a15-60b01dad03e2 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:35.453957 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74f2c8-3d79-4d4b-8a15-60b01dad03e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:35.479955 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-1f74f2c8-3d79-4d4b-8a15-60b01dad03e2 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:35.481541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-1f74f2c8-3d79-4d4b-8a15-60b01dad03e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:35.514818 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:35.581371 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 897/1779] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:35.592756 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2095b-d786-4e7f-82d6-c204dc3bf8bb None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:35.593078 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2095b-d786-4e7f-82d6-c204dc3bf8bb None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:35.593544 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2095b-d786-4e7f-82d6-c204dc3bf8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:35.619780 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-e4f2095b-d786-4e7f-82d6-c204dc3bf8bb None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:35.623339 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-e4f2095b-d786-4e7f-82d6-c204dc3bf8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:35.636107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 883/1780] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:35.648955 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74f2c8-3d79-4d4b-8a15-60b01dad03e2 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:35.651918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74f2c8-3d79-4d4b-8a15-60b01dad03e2 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:35.672993 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd461e8-0143-471d-a25b-b9346b27c9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:35.677071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd461e8-0143-471d-a25b-b9346b27c9c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:35.677716 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd461e8-0143-471d-a25b-b9346b27c9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:35.678203 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd461e8-0143-471d-a25b-b9346b27c9c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:35.679310 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd461e8-0143-471d-a25b-b9346b27c9c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:36:35.681328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd461e8-0143-471d-a25b-b9346b27c9c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:35.685732 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd461e8-0143-471d-a25b-b9346b27c9c1 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:35.709786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 898/1781] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:36:35.714336 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2095b-d786-4e7f-82d6-c204dc3bf8bb None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:35.724699 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2095b-d786-4e7f-82d6-c204dc3bf8bb None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:35.729801 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822038e-0abe-41bb-a6a7-35f0b63d3155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:35.730452 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822038e-0abe-41bb-a6a7-35f0b63d3155 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:35.730890 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822038e-0abe-41bb-a6a7-35f0b63d3155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:35.731174 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822038e-0abe-41bb-a6a7-35f0b63d3155 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:35.731949 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822038e-0abe-41bb-a6a7-35f0b63d3155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:35.734063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822038e-0abe-41bb-a6a7-35f0b63d3155 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:35.773421 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 884/1782] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:35 2022] POST /identity/v3/projects => generated 369 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:36:35.778044 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd461e8-0143-471d-a25b-b9346b27c9c1 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:35.795387 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd461e8-0143-471d-a25b-b9346b27c9c1 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:35.885381 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d52ce-2f04-4e2e-96dc-f60f484d8a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:35.886377 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d52ce-2f04-4e2e-96dc-f60f484d8a6a admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:35.886900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d52ce-2f04-4e2e-96dc-f60f484d8a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:35.887383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d52ce-2f04-4e2e-96dc-f60f484d8a6a admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:35.889215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d52ce-2f04-4e2e-96dc-f60f484d8a6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:35.891721 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d52ce-2f04-4e2e-96dc-f60f484d8a6a admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:35.956504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 899/1783] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:35 2022] POST /identity/v3/users => generated 285 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:35.957037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822038e-0abe-41bb-a6a7-35f0b63d3155 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:35.971520 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822038e-0abe-41bb-a6a7-35f0b63d3155 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:35.983056 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06343c-79b5-4894-a9b3-d1c228c11f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:35.984610 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06343c-79b5-4894-a9b3-d1c228c11f3c admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:35.984954 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06343c-79b5-4894-a9b3-d1c228c11f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:35.985235 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06343c-79b5-4894-a9b3-d1c228c11f3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:35.985958 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06343c-79b5-4894-a9b3-d1c228c11f3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:35.988028 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06343c-79b5-4894-a9b3-d1c228c11f3c admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:36.013530 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d52ce-2f04-4e2e-96dc-f60f484d8a6a admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:36.020854 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-dc7d52ce-2f04-4e2e-96dc-f60f484d8a6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:36.023803 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 885/1784] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:36:36.028708 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06343c-79b5-4894-a9b3-d1c228c11f3c admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:36.050897 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06343c-79b5-4894-a9b3-d1c228c11f3c admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:36.063743 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7608695-f3b2-42b4-9487-da94ab1cce07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:36.065678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7608695-f3b2-42b4-9487-da94ab1cce07 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:36.066138 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7608695-f3b2-42b4-9487-da94ab1cce07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:36.066422 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7608695-f3b2-42b4-9487-da94ab1cce07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:36.068043 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7608695-f3b2-42b4-9487-da94ab1cce07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:36.068821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7608695-f3b2-42b4-9487-da94ab1cce07 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:36.072439 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d52ce-2f04-4e2e-96dc-f60f484d8a6a admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:36.101478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e44364-575a-43df-802a-3a48efa14414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:36.105527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 886/1785] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:36.109637 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e44364-575a-43df-802a-3a48efa14414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:36.112805 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7608695-f3b2-42b4-9487-da94ab1cce07 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:36.113128 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e44364-575a-43df-802a-3a48efa14414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:36.113734 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e44364-575a-43df-802a-3a48efa14414 admin admin] PATH_INFO: `/v3/projects/c6298cb7b10d45b3bf2bce21d3847a30` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:36.130109 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e44364-575a-43df-802a-3a48efa14414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6298cb7b10d45b3bf2bce21d3847a30)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:36.133607 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7608695-f3b2-42b4-9487-da94ab1cce07 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:36.140097 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e44364-575a-43df-802a-3a48efa14414 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:36.160830 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c20eca-67b8-44f3-b6a4-6d8afd1f7511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:36.161879 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c20eca-67b8-44f3-b6a4-6d8afd1f7511 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:36.161879 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c20eca-67b8-44f3-b6a4-6d8afd1f7511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:36.168365 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c20eca-67b8-44f3-b6a4-6d8afd1f7511 admin admin] PATH_INFO: `/v3/projects/a012ca76102a44f0bdb1f852d87a6558/users/08da8092815c42ac83ee3249d84a4941/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:36.205828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c20eca-67b8-44f3-b6a4-6d8afd1f7511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a012ca76102a44f0bdb1f852d87a6558, user_id=08da8092815c42ac83ee3249d84a4941, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:36.215428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c20eca-67b8-44f3-b6a4-6d8afd1f7511 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:36.248633 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 887/1786] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:36 2022] PUT /identity/v3/projects/a012ca76102a44f0bdb1f852d87a6558/users/08da8092815c42ac83ee3249d84a4941/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:36:36.259862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e950f-9567-4885-8997-70e6d8fda5c2 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:36.260599 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e950f-9567-4885-8997-70e6d8fda5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:36.261059 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e950f-9567-4885-8997-70e6d8fda5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:36.313782 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a0e44364-575a-43df-802a-3a48efa14414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6298cb7b10d45b3bf2bce21d3847a30', 'request_id': 'req-a0e44364-575a-43df-802a-3a48efa14414'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:36.342920 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a0e44364-575a-43df-802a-3a48efa14414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6298cb7b10d45b3bf2bce21d3847a30', 'request_id': 'req-a0e44364-575a-43df-802a-3a48efa14414'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:36.347616 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-751e950f-9567-4885-8997-70e6d8fda5c2 None None] MFA Rules not processed for user `08da8092815c42ac83ee3249d84a4941`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:36.349247 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-751e950f-9567-4885-8997-70e6d8fda5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:36.355076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a0e44364-575a-43df-802a-3a48efa14414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6298cb7b10d45b3bf2bce21d3847a30', 'request_id': 'req-a0e44364-575a-43df-802a-3a48efa14414'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:36.385912 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a0e44364-575a-43df-802a-3a48efa14414 admin admin] The token cache is being invalidate because project c6298cb7b10d45b3bf2bce21d3847a30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:36.386212 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a0e44364-575a-43df-802a-3a48efa14414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:36.386934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a0e44364-575a-43df-802a-3a48efa14414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:36.389711 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 900/1787] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:36 2022] DELETE /identity/v3/projects/c6298cb7b10d45b3bf2bce21d3847a30 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:36.394514 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf22cda-115e-4857-b974-8b825e83660a None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:36.394861 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf22cda-115e-4857-b974-8b825e83660a None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:36.395146 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf22cda-115e-4857-b974-8b825e83660a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:36.457054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-dbf22cda-115e-4857-b974-8b825e83660a None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:36.461974 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-dbf22cda-115e-4857-b974-8b825e83660a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:36.511945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:36.524910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:36.525349 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 888/1788] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:36 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:36.529622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e950f-9567-4885-8997-70e6d8fda5c2 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:36.531130 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-751e950f-9567-4885-8997-70e6d8fda5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:36.570127 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 901/1789] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:36.578473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d0145-fa5d-4023-ae4b-6e90a316bb27 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:36.578827 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d0145-fa5d-4023-ae4b-6e90a316bb27 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:36.579112 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d0145-fa5d-4023-ae4b-6e90a316bb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:36.633255 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-a17d0145-fa5d-4023-ae4b-6e90a316bb27 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:36.634505 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-a17d0145-fa5d-4023-ae4b-6e90a316bb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:36.663582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 902/1790] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:36:36.667184 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d0145-fa5d-4023-ae4b-6e90a316bb27 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:36.671272 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-a17d0145-fa5d-4023-ae4b-6e90a316bb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:36.691926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d0145-fa5d-4023-ae4b-6e90a316bb27 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:36.694604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e950f-9567-4885-8997-70e6d8fda5c2 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:36.703288 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19595c-2af0-4455-a4e8-0dfa9c4bc801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:36.703604 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c5e1f-77be-43c0-94ea-463469e7e5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:36.704370 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c5e1f-77be-43c0-94ea-463469e7e5fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:36.705168 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c5e1f-77be-43c0-94ea-463469e7e5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:36.705519 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19595c-2af0-4455-a4e8-0dfa9c4bc801 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:36.705890 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c5e1f-77be-43c0-94ea-463469e7e5fc admin admin] PATH_INFO: `/v3/users/7e5cc956082d4c39ad2132086251be72` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:36.706329 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19595c-2af0-4455-a4e8-0dfa9c4bc801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:36.707086 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19595c-2af0-4455-a4e8-0dfa9c4bc801 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:36.708684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19595c-2af0-4455-a4e8-0dfa9c4bc801 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:36.708982 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c5e1f-77be-43c0-94ea-463469e7e5fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e5cc956082d4c39ad2132086251be72)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:36.711145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c5e1f-77be-43c0-94ea-463469e7e5fc admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:36.713520 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19595c-2af0-4455-a4e8-0dfa9c4bc801 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:36.745937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 889/1791] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:36 2022] POST /identity/v3/projects => generated 367 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:36.746729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19595c-2af0-4455-a4e8-0dfa9c4bc801 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:36.786041 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-b07c5e1f-77be-43c0-94ea-463469e7e5fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e5cc956082d4c39ad2132086251be72'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:36.795636 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-b07c5e1f-77be-43c0-94ea-463469e7e5fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e5cc956082d4c39ad2132086251be72'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:36.815256 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-b07c5e1f-77be-43c0-94ea-463469e7e5fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e5cc956082d4c39ad2132086251be72'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:36.819522 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19595c-2af0-4455-a4e8-0dfa9c4bc801 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:36.823313 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-b07c5e1f-77be-43c0-94ea-463469e7e5fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e5cc956082d4c39ad2132086251be72'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:36.824028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-b07c5e1f-77be-43c0-94ea-463469e7e5fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e5cc956082d4c39ad2132086251be72'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:36.824507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90bb7d-fe18-434a-b198-8dc8ac14d914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:36.825197 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90bb7d-fe18-434a-b198-8dc8ac14d914 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:36.825623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90bb7d-fe18-434a-b198-8dc8ac14d914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:36.825991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90bb7d-fe18-434a-b198-8dc8ac14d914 admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:36.826807 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90bb7d-fe18-434a-b198-8dc8ac14d914 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:36:36.827130 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90bb7d-fe18-434a-b198-8dc8ac14d914 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:36.828855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90bb7d-fe18-434a-b198-8dc8ac14d914 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:36.831752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 903/1792] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:36 2022] DELETE /identity/v3/users/7e5cc956082d4c39ad2132086251be72 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:36.832657 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c5e1f-77be-43c0-94ea-463469e7e5fc admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:36.834098 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b07c5e1f-77be-43c0-94ea-463469e7e5fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:36.842623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 890/1793] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:36:36.843982 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada2442-150a-4893-b2c4-c83ffc705f1f None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:36.844270 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada2442-150a-4893-b2c4-c83ffc705f1f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:36.844543 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada2442-150a-4893-b2c4-c83ffc705f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:36.865169 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-9ada2442-150a-4893-b2c4-c83ffc705f1f None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:36.866474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9ada2442-150a-4893-b2c4-c83ffc705f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:36.910842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c5e1f-77be-43c0-94ea-463469e7e5fc admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:36.915826 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254c76a-7531-4430-bcbb-ff5ef8032c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:36.916532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254c76a-7531-4430-bcbb-ff5ef8032c38 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:36.916826 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254c76a-7531-4430-bcbb-ff5ef8032c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:36.917126 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254c76a-7531-4430-bcbb-ff5ef8032c38 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:36.917791 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254c76a-7531-4430-bcbb-ff5ef8032c38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:36.920187 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254c76a-7531-4430-bcbb-ff5ef8032c38 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:36.936606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 891/1794] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:36.937543 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada2442-150a-4893-b2c4-c83ffc705f1f None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:36.941074 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9ada2442-150a-4893-b2c4-c83ffc705f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:36.961282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada2442-150a-4893-b2c4-c83ffc705f1f None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:36.962484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 904/1795] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:36 2022] POST /identity/v3/users => generated 290 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:36.963603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254c76a-7531-4430-bcbb-ff5ef8032c38 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:36.967691 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-6254c76a-7531-4430-bcbb-ff5ef8032c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:36.975851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c09c0-a16a-4881-8e3f-88a38218fbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:36.976635 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c09c0-a16a-4881-8e3f-88a38218fbba admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:36.976814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c09c0-a16a-4881-8e3f-88a38218fbba admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:36.977146 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c09c0-a16a-4881-8e3f-88a38218fbba admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:36.977813 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c09c0-a16a-4881-8e3f-88a38218fbba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:36.980016 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c09c0-a16a-4881-8e3f-88a38218fbba admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:37.003237 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:37.005174 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:37.016711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 892/1796] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:36 2022] POST /identity/v3/projects => generated 365 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:37.016711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c09c0-a16a-4881-8e3f-88a38218fbba admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:37.037600 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254c76a-7531-4430-bcbb-ff5ef8032c38 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:37.083614 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c09c0-a16a-4881-8e3f-88a38218fbba admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:37.089265 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c0e21-b9b2-4552-80df-7583c0bebfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:37.090012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c0e21-b9b2-4552-80df-7583c0bebfe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:37.090988 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c0e21-b9b2-4552-80df-7583c0bebfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:37.091866 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c0e21-b9b2-4552-80df-7583c0bebfe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:37.092664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c0e21-b9b2-4552-80df-7583c0bebfe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:37.094408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c0e21-b9b2-4552-80df-7583c0bebfe3 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:37.095144 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58642b6-3416-4a86-a608-580a941c27db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:37.095962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58642b6-3416-4a86-a608-580a941c27db service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:37.096311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58642b6-3416-4a86-a608-580a941c27db service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:37.096654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58642b6-3416-4a86-a608-580a941c27db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:37.105324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 893/1797] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:37.106468 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58642b6-3416-4a86-a608-580a941c27db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:37.108592 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58642b6-3416-4a86-a608-580a941c27db service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:37.110029 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c0e21-b9b2-4552-80df-7583c0bebfe3 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:37.119108 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 905/1798] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:37.120069 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c0e21-b9b2-4552-80df-7583c0bebfe3 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:37.120392 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58642b6-3416-4a86-a608-580a941c27db service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:37.125244 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614287f-eba2-4ef2-8dd2-a057b3510fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:37.125911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614287f-eba2-4ef2-8dd2-a057b3510fbc admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:37.126205 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614287f-eba2-4ef2-8dd2-a057b3510fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:37.126551 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614287f-eba2-4ef2-8dd2-a057b3510fbc admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:37.127230 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614287f-eba2-4ef2-8dd2-a057b3510fbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:37.129194 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614287f-eba2-4ef2-8dd2-a057b3510fbc admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:37.134463 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58642b6-3416-4a86-a608-580a941c27db service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:37.139669 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511bc9a8-cca8-4ce6-a2d3-af2838a73576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:37.140449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bc9a8-cca8-4ce6-a2d3-af2838a73576 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:37.140609 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bc9a8-cca8-4ce6-a2d3-af2838a73576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:37.140932 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bc9a8-cca8-4ce6-a2d3-af2838a73576 admin admin] PATH_INFO: `/v3/projects/d536cd0221ff4bf78bf0d7e65bc20e96/users/b5363093880a4c689d30dfcb61d37660/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:37.154887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 894/1799] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:37 2022] POST /identity/v3/users => generated 283 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:37.158291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511bc9a8-cca8-4ce6-a2d3-af2838a73576 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d536cd0221ff4bf78bf0d7e65bc20e96, user_id=b5363093880a4c689d30dfcb61d37660, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:37.158433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614287f-eba2-4ef2-8dd2-a057b3510fbc admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:37.162012 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511bc9a8-cca8-4ce6-a2d3-af2838a73576 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:37.172169 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614287f-eba2-4ef2-8dd2-a057b3510fbc admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:37.178067 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb925485-6f25-4d2e-bf6d-8048d29fb7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:37.178966 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb925485-6f25-4d2e-bf6d-8048d29fb7c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:37.179265 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb925485-6f25-4d2e-bf6d-8048d29fb7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:37.179613 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb925485-6f25-4d2e-bf6d-8048d29fb7c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:37.180398 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb925485-6f25-4d2e-bf6d-8048d29fb7c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:37.181692 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 906/1800] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:37 2022] PUT /identity/v3/projects/d536cd0221ff4bf78bf0d7e65bc20e96/users/b5363093880a4c689d30dfcb61d37660/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:37.182037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb925485-6f25-4d2e-bf6d-8048d29fb7c7 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:37.186046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511bc9a8-cca8-4ce6-a2d3-af2838a73576 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:37.194649 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 895/1801] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:37.198401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb925485-6f25-4d2e-bf6d-8048d29fb7c7 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:37.209689 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb925485-6f25-4d2e-bf6d-8048d29fb7c7 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:37.260439 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511bc9a8-cca8-4ce6-a2d3-af2838a73576 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:37.266613 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44693af5-19e5-45d4-830a-78b8816bfff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:37.267688 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44693af5-19e5-45d4-830a-78b8816bfff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:37.268115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44693af5-19e5-45d4-830a-78b8816bfff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:37.268530 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44693af5-19e5-45d4-830a-78b8816bfff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:37.269657 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44693af5-19e5-45d4-830a-78b8816bfff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:37.270927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0029df-2954-4934-9e79-ee98619cf2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:37.272442 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44693af5-19e5-45d4-830a-78b8816bfff6 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:37.272834 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0029df-2954-4934-9e79-ee98619cf2c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:37.273170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0029df-2954-4934-9e79-ee98619cf2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:37.273446 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0029df-2954-4934-9e79-ee98619cf2c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:37.274881 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0029df-2954-4934-9e79-ee98619cf2c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:37.280700 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0029df-2954-4934-9e79-ee98619cf2c1 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:37.290988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 907/1802] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:36:37.292001 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 896/1803] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:36:37.295032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44693af5-19e5-45d4-830a-78b8816bfff6 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:37.296293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0029df-2954-4934-9e79-ee98619cf2c1 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:37.306612 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0029df-2954-4934-9e79-ee98619cf2c1 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:37.307221 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44693af5-19e5-45d4-830a-78b8816bfff6 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:37.312171 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb86a3-0ad1-41f9-8c29-6260e6bd74a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:37.312511 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496f0b2-094b-41e1-a15d-e3d7013256b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:37.315278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb86a3-0ad1-41f9-8c29-6260e6bd74a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:37.315400 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496f0b2-094b-41e1-a15d-e3d7013256b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:37.315489 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb86a3-0ad1-41f9-8c29-6260e6bd74a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:37.315572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496f0b2-094b-41e1-a15d-e3d7013256b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:37.315654 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb86a3-0ad1-41f9-8c29-6260e6bd74a4 admin admin] PATH_INFO: `/v3/projects/f96a0f7c8bf44b0f880adc49a2039128/users/a364dff0921f4416a78827a61f4477de/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:37.315753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496f0b2-094b-41e1-a15d-e3d7013256b4 admin admin] PATH_INFO: `/v3/projects/d536cd0221ff4bf78bf0d7e65bc20e96/users/b5363093880a4c689d30dfcb61d37660/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:37.318057 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9496f0b2-094b-41e1-a15d-e3d7013256b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d536cd0221ff4bf78bf0d7e65bc20e96, user_id=b5363093880a4c689d30dfcb61d37660, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:37.320050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9496f0b2-094b-41e1-a15d-e3d7013256b4 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:37.330775 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb86a3-0ad1-41f9-8c29-6260e6bd74a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96a0f7c8bf44b0f880adc49a2039128, user_id=a364dff0921f4416a78827a61f4477de, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:37.333572 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb86a3-0ad1-41f9-8c29-6260e6bd74a4 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:37.334792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 908/1804] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:37 2022] PUT /identity/v3/projects/d536cd0221ff4bf78bf0d7e65bc20e96/users/b5363093880a4c689d30dfcb61d37660/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:37.346238 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4ca82-bdf3-4948-a783-0d2d6297b069 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:37.346567 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4ca82-bdf3-4948-a783-0d2d6297b069 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:37.346872 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4ca82-bdf3-4948-a783-0d2d6297b069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:37.356582 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 897/1805] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:37 2022] PUT /identity/v3/projects/f96a0f7c8bf44b0f880adc49a2039128/users/a364dff0921f4416a78827a61f4477de/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:37.368077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbaa8a-2450-4f73-b619-d322e77e1a5e None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:37.368363 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbaa8a-2450-4f73-b619-d322e77e1a5e None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:37.368638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbaa8a-2450-4f73-b619-d322e77e1a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:37.378943 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-e7e4ca82-bdf3-4948-a783-0d2d6297b069 None None] MFA Rules not processed for user `b5363093880a4c689d30dfcb61d37660`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:37.380454 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e7e4ca82-bdf3-4948-a783-0d2d6297b069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:37.414342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-f5cbaa8a-2450-4f73-b619-d322e77e1a5e None None] MFA Rules not processed for user `a364dff0921f4416a78827a61f4477de`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:37.415646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-f5cbaa8a-2450-4f73-b619-d322e77e1a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:37.424156 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:37.454411 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:37.497566 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 909/1806] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:37 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:37.499642 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4ca82-bdf3-4948-a783-0d2d6297b069 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:37.501349 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e7e4ca82-bdf3-4948-a783-0d2d6297b069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:37.531778 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 898/1807] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:37 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:36:37.537094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbaa8a-2450-4f73-b619-d322e77e1a5e None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:37.538591 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-f5cbaa8a-2450-4f73-b619-d322e77e1a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:37.605839 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4ca82-bdf3-4948-a783-0d2d6297b069 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:37.615204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdd007-bb50-459f-998e-db4c4a911349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:37.616450 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdd007-bb50-459f-998e-db4c4a911349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:37.616753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdd007-bb50-459f-998e-db4c4a911349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:37.617445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdd007-bb50-459f-998e-db4c4a911349 admin admin] PATH_INFO: `/v3/projects/017e62434ed04940be72828ee8e66929` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:37.618522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdd007-bb50-459f-998e-db4c4a911349 admin admin] RBAC: Authorizing `identity:delete_project(project_id=017e62434ed04940be72828ee8e66929)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:37.620613 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdd007-bb50-459f-998e-db4c4a911349 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:37.639893 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbaa8a-2450-4f73-b619-d322e77e1a5e None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:37.646087 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3aa3f6-8eb9-44f5-9300-40e173532a41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:37.646732 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3aa3f6-8eb9-44f5-9300-40e173532a41 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:37.647028 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3aa3f6-8eb9-44f5-9300-40e173532a41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:37.647307 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3aa3f6-8eb9-44f5-9300-40e173532a41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:37.658513 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3aa3f6-8eb9-44f5-9300-40e173532a41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:37.660664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3aa3f6-8eb9-44f5-9300-40e173532a41 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:37.675257 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 899/1808] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:37 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:37.676854 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3aa3f6-8eb9-44f5-9300-40e173532a41 service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:37.713631 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f9fdd007-bb50-459f-998e-db4c4a911349 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '017e62434ed04940be72828ee8e66929', 'request_id': 'req-f9fdd007-bb50-459f-998e-db4c4a911349'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:37.718100 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3aa3f6-8eb9-44f5-9300-40e173532a41 service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:37.730418 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f9fdd007-bb50-459f-998e-db4c4a911349 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '017e62434ed04940be72828ee8e66929', 'request_id': 'req-f9fdd007-bb50-459f-998e-db4c4a911349'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:37.737498 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f9fdd007-bb50-459f-998e-db4c4a911349 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '017e62434ed04940be72828ee8e66929', 'request_id': 'req-f9fdd007-bb50-459f-998e-db4c4a911349'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:37.750455 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f9fdd007-bb50-459f-998e-db4c4a911349 admin admin] The token cache is being invalidate because project 017e62434ed04940be72828ee8e66929 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:37.750782 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f9fdd007-bb50-459f-998e-db4c4a911349 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:37.751457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f9fdd007-bb50-459f-998e-db4c4a911349 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:37.753103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 910/1809] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:37 2022] DELETE /identity/v3/projects/017e62434ed04940be72828ee8e66929 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:37.753866 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdd007-bb50-459f-998e-db4c4a911349 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:37.755200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f9fdd007-bb50-459f-998e-db4c4a911349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:37.805570 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afe84e-72c4-4d46-96b8-342d25b3a297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:37.806510 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afe84e-72c4-4d46-96b8-342d25b3a297 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:37.806992 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afe84e-72c4-4d46-96b8-342d25b3a297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:37.807459 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afe84e-72c4-4d46-96b8-342d25b3a297 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:37.808328 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75afe84e-72c4-4d46-96b8-342d25b3a297 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:37.811501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75afe84e-72c4-4d46-96b8-342d25b3a297 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:37.828226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdd007-bb50-459f-998e-db4c4a911349 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:37.835158 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409c956-c457-441c-be8d-a5352bc7c139 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:37.835795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409c956-c457-441c-be8d-a5352bc7c139 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:37.836076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409c956-c457-441c-be8d-a5352bc7c139 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:37.836349 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409c956-c457-441c-be8d-a5352bc7c139 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:37.841571 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f409c956-c457-441c-be8d-a5352bc7c139 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:37.864854 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 900/1810] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:37 2022] POST /identity/v3/projects => generated 363 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:37.864854 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afe84e-72c4-4d46-96b8-342d25b3a297 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:37.865191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-75afe84e-72c4-4d46-96b8-342d25b3a297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:37.866178 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:37.943560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409c956-c457-441c-be8d-a5352bc7c139 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:37.957577 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409c956-c457-441c-be8d-a5352bc7c139 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:37.973094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:37.980727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afe84e-72c4-4d46-96b8-342d25b3a297 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:37.989463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b6448-365f-4cb5-86c6-72873eb68740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:37.989636 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 911/1811] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:37 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:37.990449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409c956-c457-441c-be8d-a5352bc7c139 service cinder] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:37.991166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b6448-365f-4cb5-86c6-72873eb68740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:37.991463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b6448-365f-4cb5-86c6-72873eb68740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:37.991739 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b6448-365f-4cb5-86c6-72873eb68740 admin admin] PATH_INFO: `/v3/users/1823867f9332447185a1eddbdec13663` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:37.993047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f409c956-c457-441c-be8d-a5352bc7c139 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:37.993169 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204b6448-365f-4cb5-86c6-72873eb68740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1823867f9332447185a1eddbdec13663)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:37.996002 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204b6448-365f-4cb5-86c6-72873eb68740 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:38.012590 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409c956-c457-441c-be8d-a5352bc7c139 service cinder] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:38.022628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c4b26-32c6-4f28-b113-5753d70cbd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:38.023416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c4b26-32c6-4f28-b113-5753d70cbd4d admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:38.023755 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c4b26-32c6-4f28-b113-5753d70cbd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:38.024063 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c4b26-32c6-4f28-b113-5753d70cbd4d admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:38.024754 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961c4b26-32c6-4f28-b113-5753d70cbd4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:38.027095 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961c4b26-32c6-4f28-b113-5753d70cbd4d admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:38.073033 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 912/1812] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:37 2022] POST /identity/v3/users => generated 288 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:38.079653 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c4b26-32c6-4f28-b113-5753d70cbd4d admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:38.100233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c4b26-32c6-4f28-b113-5753d70cbd4d admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:38.104227 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-204b6448-365f-4cb5-86c6-72873eb68740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1823867f9332447185a1eddbdec13663'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:38.113825 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ba96a-9cde-4448-af4f-ef8ce2838436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:38.114841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ba96a-9cde-4448-af4f-ef8ce2838436 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:38.117231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-204b6448-365f-4cb5-86c6-72873eb68740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1823867f9332447185a1eddbdec13663'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:38.118782 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ba96a-9cde-4448-af4f-ef8ce2838436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:38.119186 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ba96a-9cde-4448-af4f-ef8ce2838436 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:38.119931 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ba96a-9cde-4448-af4f-ef8ce2838436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:38.122565 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ba96a-9cde-4448-af4f-ef8ce2838436 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:38.132220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-204b6448-365f-4cb5-86c6-72873eb68740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1823867f9332447185a1eddbdec13663'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:38.149942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 913/1813] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:38.150130 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-204b6448-365f-4cb5-86c6-72873eb68740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1823867f9332447185a1eddbdec13663'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:38.150130 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-204b6448-365f-4cb5-86c6-72873eb68740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1823867f9332447185a1eddbdec13663'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:38.154995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ba96a-9cde-4448-af4f-ef8ce2838436 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:38.159429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-0d9ba96a-9cde-4448-af4f-ef8ce2838436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:38.176333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 901/1814] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:37 2022] DELETE /identity/v3/users/1823867f9332447185a1eddbdec13663 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:38.224712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ba96a-9cde-4448-af4f-ef8ce2838436 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:38.333917 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47886b5-7108-4224-b92a-22069e9ea5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:38.334725 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47886b5-7108-4224-b92a-22069e9ea5dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:38.335107 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47886b5-7108-4224-b92a-22069e9ea5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:38.335482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47886b5-7108-4224-b92a-22069e9ea5dc admin admin] PATH_INFO: `/v3/projects/ccd7282e500d4ddfa7fa0057fc1fab6c/users/26cd152e660e4e3d9996cabf7d18cdab/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:38.371775 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47886b5-7108-4224-b92a-22069e9ea5dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccd7282e500d4ddfa7fa0057fc1fab6c, user_id=26cd152e660e4e3d9996cabf7d18cdab, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:38.381226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47886b5-7108-4224-b92a-22069e9ea5dc admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:38.448708 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b6448-365f-4cb5-86c6-72873eb68740 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:38.449307 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 914/1815] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:38 2022] PUT /identity/v3/projects/ccd7282e500d4ddfa7fa0057fc1fab6c/users/26cd152e660e4e3d9996cabf7d18cdab/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:38.463676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b6448-365f-4cb5-86c6-72873eb68740 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:38.548434 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8dc25e-c961-43ba-b61e-9d27555992ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:38.549284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8dc25e-c961-43ba-b61e-9d27555992ad admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:38.549658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8dc25e-c961-43ba-b61e-9d27555992ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:38.549947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8dc25e-c961-43ba-b61e-9d27555992ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:38.550658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8dc25e-c961-43ba-b61e-9d27555992ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:38.552337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8dc25e-c961-43ba-b61e-9d27555992ad admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:38.569278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 902/1816] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:36:38.571103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47886b5-7108-4224-b92a-22069e9ea5dc admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:38.586626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47886b5-7108-4224-b92a-22069e9ea5dc admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:38.593763 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818cdb97-9d84-4dfb-91cb-cb8b1363acf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:38.594804 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cdb97-9d84-4dfb-91cb-cb8b1363acf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:38.595352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cdb97-9d84-4dfb-91cb-cb8b1363acf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:38.595824 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cdb97-9d84-4dfb-91cb-cb8b1363acf6 admin admin] PATH_INFO: `/v3/projects/ccd7282e500d4ddfa7fa0057fc1fab6c/users/26cd152e660e4e3d9996cabf7d18cdab/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:38.598428 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818cdb97-9d84-4dfb-91cb-cb8b1363acf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccd7282e500d4ddfa7fa0057fc1fab6c, user_id=26cd152e660e4e3d9996cabf7d18cdab, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:38.601557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818cdb97-9d84-4dfb-91cb-cb8b1363acf6 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:38.614634 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 915/1817] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:38 2022] PUT /identity/v3/projects/ccd7282e500d4ddfa7fa0057fc1fab6c/users/26cd152e660e4e3d9996cabf7d18cdab/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:38.625716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d887de-7215-4e99-8b6e-7c0f90b83710 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:38.626070 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d887de-7215-4e99-8b6e-7c0f90b83710 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:38.626355 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d887de-7215-4e99-8b6e-7c0f90b83710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:38.690007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-c8d887de-7215-4e99-8b6e-7c0f90b83710 None None] MFA Rules not processed for user `26cd152e660e4e3d9996cabf7d18cdab`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:38.691783 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c8d887de-7215-4e99-8b6e-7c0f90b83710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:38.772068 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:38.787753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 903/1818] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:38 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:36:38.802693 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818cdb97-9d84-4dfb-91cb-cb8b1363acf6 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:38.804371 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-818cdb97-9d84-4dfb-91cb-cb8b1363acf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:38.873144 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818cdb97-9d84-4dfb-91cb-cb8b1363acf6 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:38.877582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cda7e5-f73e-49bb-8a85-3c52db0c33b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:38.878227 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cda7e5-f73e-49bb-8a85-3c52db0c33b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:38.878516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cda7e5-f73e-49bb-8a85-3c52db0c33b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:38.878795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cda7e5-f73e-49bb-8a85-3c52db0c33b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:38.896372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cda7e5-f73e-49bb-8a85-3c52db0c33b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:38.898023 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cda7e5-f73e-49bb-8a85-3c52db0c33b2 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:38.902611 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:38.912676 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 916/1819] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:38 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:39.491192 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d887de-7215-4e99-8b6e-7c0f90b83710 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:39.674591 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d887de-7215-4e99-8b6e-7c0f90b83710 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:39.682312 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c36dc-8512-41f1-b8e4-30a0440daccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:39.683472 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c36dc-8512-41f1-b8e4-30a0440daccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:39.684043 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c36dc-8512-41f1-b8e4-30a0440daccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:39.684474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c36dc-8512-41f1-b8e4-30a0440daccc admin admin] PATH_INFO: `/v3/users/a364dff0921f4416a78827a61f4477de` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:39.687328 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c36dc-8512-41f1-b8e4-30a0440daccc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a364dff0921f4416a78827a61f4477de)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:39.689667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c36dc-8512-41f1-b8e4-30a0440daccc admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:39.766724 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-f93c36dc-8512-41f1-b8e4-30a0440daccc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a364dff0921f4416a78827a61f4477de'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:39.781084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-f93c36dc-8512-41f1-b8e4-30a0440daccc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a364dff0921f4416a78827a61f4477de'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:39.796039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-f93c36dc-8512-41f1-b8e4-30a0440daccc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a364dff0921f4416a78827a61f4477de'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:39.821546 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-f93c36dc-8512-41f1-b8e4-30a0440daccc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a364dff0921f4416a78827a61f4477de'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:39.822843 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-f93c36dc-8512-41f1-b8e4-30a0440daccc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a364dff0921f4416a78827a61f4477de'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:39.840669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 904/1820] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:39 2022] DELETE /identity/v3/users/a364dff0921f4416a78827a61f4477de => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:39.845415 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cb573-d971-4e44-bee0-c1b51ec45435 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:39.846316 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cb573-d971-4e44-bee0-c1b51ec45435 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:39.846783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cb573-d971-4e44-bee0-c1b51ec45435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:39.868247 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c36dc-8512-41f1-b8e4-30a0440daccc admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:39.870774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-f93c36dc-8512-41f1-b8e4-30a0440daccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:39.884829 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-a48cb573-d971-4e44-bee0-c1b51ec45435 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:39.886311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-a48cb573-d971-4e44-bee0-c1b51ec45435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:39.984013 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 917/1821] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:39.996374 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48cb573-d971-4e44-bee0-c1b51ec45435 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:39.999733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-a48cb573-d971-4e44-bee0-c1b51ec45435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:40.033436 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c36dc-8512-41f1-b8e4-30a0440daccc admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:40.052222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705582d-d214-46f6-8cc1-3e942ced3936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:40.053145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705582d-d214-46f6-8cc1-3e942ced3936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:40.053590 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705582d-d214-46f6-8cc1-3e942ced3936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:40.054016 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705582d-d214-46f6-8cc1-3e942ced3936 admin admin] PATH_INFO: `/v3/projects/d48fee9d68394a208d73505a7590c186` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:40.056066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705582d-d214-46f6-8cc1-3e942ced3936 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d48fee9d68394a208d73505a7590c186)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:40.062052 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705582d-d214-46f6-8cc1-3e942ced3936 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:40.098045 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:40.130076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48cb573-d971-4e44-bee0-c1b51ec45435 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:40.139705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c6005-2e4a-43a3-a7c4-2f65bfe4b6b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:40.141195 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c6005-2e4a-43a3-a7c4-2f65bfe4b6b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:40.141514 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c6005-2e4a-43a3-a7c4-2f65bfe4b6b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:40.141787 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c6005-2e4a-43a3-a7c4-2f65bfe4b6b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:40.159391 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1c6005-2e4a-43a3-a7c4-2f65bfe4b6b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:40.162326 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1c6005-2e4a-43a3-a7c4-2f65bfe4b6b1 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:40.176982 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 918/1822] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:40.183854 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b705582d-d214-46f6-8cc1-3e942ced3936 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd48fee9d68394a208d73505a7590c186', 'request_id': 'req-b705582d-d214-46f6-8cc1-3e942ced3936'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:40.199485 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b705582d-d214-46f6-8cc1-3e942ced3936 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd48fee9d68394a208d73505a7590c186', 'request_id': 'req-b705582d-d214-46f6-8cc1-3e942ced3936'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:40.213970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b705582d-d214-46f6-8cc1-3e942ced3936 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd48fee9d68394a208d73505a7590c186', 'request_id': 'req-b705582d-d214-46f6-8cc1-3e942ced3936'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:40.270041 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b705582d-d214-46f6-8cc1-3e942ced3936 admin admin] The token cache is being invalidate because project d48fee9d68394a208d73505a7590c186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:40.270645 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b705582d-d214-46f6-8cc1-3e942ced3936 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:40.272469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b705582d-d214-46f6-8cc1-3e942ced3936 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:40.280708 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 905/1823] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:39 2022] DELETE /identity/v3/projects/d48fee9d68394a208d73505a7590c186 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:40.281849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c6005-2e4a-43a3-a7c4-2f65bfe4b6b1 service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:40.314281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c6005-2e4a-43a3-a7c4-2f65bfe4b6b1 service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:40.431474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181df6f-129f-47ed-a959-b2917c2c4ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:40.432255 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181df6f-129f-47ed-a959-b2917c2c4ebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:40.445839 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181df6f-129f-47ed-a959-b2917c2c4ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:40.446246 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181df6f-129f-47ed-a959-b2917c2c4ebb admin admin] PATH_INFO: `/v3/users/ffa78cd64fde4e12baeaadb9debcd808` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:40.462161 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181df6f-129f-47ed-a959-b2917c2c4ebb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffa78cd64fde4e12baeaadb9debcd808)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:40.468956 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181df6f-129f-47ed-a959-b2917c2c4ebb admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:40.543916 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:40.547381 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-9181df6f-129f-47ed-a959-b2917c2c4ebb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffa78cd64fde4e12baeaadb9debcd808'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:40.557275 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-9181df6f-129f-47ed-a959-b2917c2c4ebb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffa78cd64fde4e12baeaadb9debcd808'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:40.570941 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-9181df6f-129f-47ed-a959-b2917c2c4ebb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffa78cd64fde4e12baeaadb9debcd808'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:40.579508 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-9181df6f-129f-47ed-a959-b2917c2c4ebb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffa78cd64fde4e12baeaadb9debcd808'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:40.580261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-9181df6f-129f-47ed-a959-b2917c2c4ebb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffa78cd64fde4e12baeaadb9debcd808'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:40.588848 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 919/1824] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:40 2022] DELETE /identity/v3/users/ffa78cd64fde4e12baeaadb9debcd808 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:40.894326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705582d-d214-46f6-8cc1-3e942ced3936 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:40.895907 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b705582d-d214-46f6-8cc1-3e942ced3936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:40.912643 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705582d-d214-46f6-8cc1-3e942ced3936 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:40.985442 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d96fc-d7b6-4b23-8a7a-2d521597ef8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:40.986196 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d96fc-d7b6-4b23-8a7a-2d521597ef8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:40.986772 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d96fc-d7b6-4b23-8a7a-2d521597ef8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:40.987117 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d96fc-d7b6-4b23-8a7a-2d521597ef8d admin admin] PATH_INFO: `/v3/projects/f96a0f7c8bf44b0f880adc49a2039128` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:40.988376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d96fc-d7b6-4b23-8a7a-2d521597ef8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96a0f7c8bf44b0f880adc49a2039128)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:40.988723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181df6f-129f-47ed-a959-b2917c2c4ebb admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:40.990250 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-9181df6f-129f-47ed-a959-b2917c2c4ebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:40.990679 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d96fc-d7b6-4b23-8a7a-2d521597ef8d admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:41.004635 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181df6f-129f-47ed-a959-b2917c2c4ebb admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:41.009936 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f205a-f1c0-4aa9-be50-2ec1872bea82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:41.010566 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f205a-f1c0-4aa9-be50-2ec1872bea82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:41.010926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f205a-f1c0-4aa9-be50-2ec1872bea82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:41.011283 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f205a-f1c0-4aa9-be50-2ec1872bea82 admin admin] PATH_INFO: `/v3/projects/9c3dc65274504d0888c83d256f29c52c` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:41.019950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f205a-f1c0-4aa9-be50-2ec1872bea82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c3dc65274504d0888c83d256f29c52c)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:41.022548 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f205a-f1c0-4aa9-be50-2ec1872bea82 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:41.080926 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-9f2d96fc-d7b6-4b23-8a7a-2d521597ef8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96a0f7c8bf44b0f880adc49a2039128', 'request_id': 'req-9f2d96fc-d7b6-4b23-8a7a-2d521597ef8d'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:41.092181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-9f2d96fc-d7b6-4b23-8a7a-2d521597ef8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96a0f7c8bf44b0f880adc49a2039128', 'request_id': 'req-9f2d96fc-d7b6-4b23-8a7a-2d521597ef8d'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:41.104095 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-907f205a-f1c0-4aa9-be50-2ec1872bea82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c3dc65274504d0888c83d256f29c52c', 'request_id': 'req-907f205a-f1c0-4aa9-be50-2ec1872bea82'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:41.107082 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-9f2d96fc-d7b6-4b23-8a7a-2d521597ef8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96a0f7c8bf44b0f880adc49a2039128', 'request_id': 'req-9f2d96fc-d7b6-4b23-8a7a-2d521597ef8d'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:41.114017 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-907f205a-f1c0-4aa9-be50-2ec1872bea82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c3dc65274504d0888c83d256f29c52c', 'request_id': 'req-907f205a-f1c0-4aa9-be50-2ec1872bea82'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:41.120487 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-9f2d96fc-d7b6-4b23-8a7a-2d521597ef8d admin admin] The token cache is being invalidate because project f96a0f7c8bf44b0f880adc49a2039128 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:41.120784 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-9f2d96fc-d7b6-4b23-8a7a-2d521597ef8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:41.121623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-9f2d96fc-d7b6-4b23-8a7a-2d521597ef8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:41.123233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-907f205a-f1c0-4aa9-be50-2ec1872bea82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c3dc65274504d0888c83d256f29c52c', 'request_id': 'req-907f205a-f1c0-4aa9-be50-2ec1872bea82'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:41.123752 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 906/1825] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:40 2022] DELETE /identity/v3/projects/f96a0f7c8bf44b0f880adc49a2039128 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:41.127866 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d96fc-d7b6-4b23-8a7a-2d521597ef8d admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:41.129609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9f2d96fc-d7b6-4b23-8a7a-2d521597ef8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:41.140001 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-907f205a-f1c0-4aa9-be50-2ec1872bea82 admin admin] The token cache is being invalidate because project 9c3dc65274504d0888c83d256f29c52c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:41.140380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-907f205a-f1c0-4aa9-be50-2ec1872bea82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:41.141293 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-907f205a-f1c0-4aa9-be50-2ec1872bea82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:41.143766 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 920/1826] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:40 2022] DELETE /identity/v3/projects/9c3dc65274504d0888c83d256f29c52c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:41.154722 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127686f6-88db-4855-b65a-e55486e1b4b0 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:41.155009 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127686f6-88db-4855-b65a-e55486e1b4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:41.155288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127686f6-88db-4855-b65a-e55486e1b4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:41.189387 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-127686f6-88db-4855-b65a-e55486e1b4b0 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:41.190744 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-127686f6-88db-4855-b65a-e55486e1b4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:41.210312 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:41.241893 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d96fc-d7b6-4b23-8a7a-2d521597ef8d admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:41.264913 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 921/1827] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:41.266360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1da2d1-3260-42f9-baf0-11c177a7c769 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:41.266680 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1da2d1-3260-42f9-baf0-11c177a7c769 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:41.266957 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1da2d1-3260-42f9-baf0-11c177a7c769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:41.293070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-1d1da2d1-3260-42f9-baf0-11c177a7c769 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:41.295099 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-1d1da2d1-3260-42f9-baf0-11c177a7c769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:41.303661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 922/1828] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:36:41.307662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1da2d1-3260-42f9-baf0-11c177a7c769 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:41.313595 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9756b72-ef73-4030-8cfb-de9250e707b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:41.314534 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9756b72-ef73-4030-8cfb-de9250e707b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:41.314822 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9756b72-ef73-4030-8cfb-de9250e707b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:41.315097 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9756b72-ef73-4030-8cfb-de9250e707b2 admin admin] PATH_INFO: `/v3/users/26cd152e660e4e3d9996cabf7d18cdab` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:41.316291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9756b72-ef73-4030-8cfb-de9250e707b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26cd152e660e4e3d9996cabf7d18cdab)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:41.318006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1da2d1-3260-42f9-baf0-11c177a7c769 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:41.318266 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9756b72-ef73-4030-8cfb-de9250e707b2 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:41.322910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d713a3-33e5-4e09-9ed6-883bc6c6fa36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:41.323566 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d713a3-33e5-4e09-9ed6-883bc6c6fa36 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:41.323846 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d713a3-33e5-4e09-9ed6-883bc6c6fa36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:41.324931 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d713a3-33e5-4e09-9ed6-883bc6c6fa36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:41.324931 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d713a3-33e5-4e09-9ed6-883bc6c6fa36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:36:41.325160 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d713a3-33e5-4e09-9ed6-883bc6c6fa36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:41.327852 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d713a3-33e5-4e09-9ed6-883bc6c6fa36 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:41.344548 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 923/1829] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:36:41.350667 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d713a3-33e5-4e09-9ed6-883bc6c6fa36 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:41.362528 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d713a3-33e5-4e09-9ed6-883bc6c6fa36 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:41.367812 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0682c-e17e-4e2f-afc7-68d6fdde8bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:41.368582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0682c-e17e-4e2f-afc7-68d6fdde8bf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:41.368991 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0682c-e17e-4e2f-afc7-68d6fdde8bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:41.369370 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0682c-e17e-4e2f-afc7-68d6fdde8bf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:41.370151 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0682c-e17e-4e2f-afc7-68d6fdde8bf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:41.372158 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0682c-e17e-4e2f-afc7-68d6fdde8bf9 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:41.379919 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b9756b72-ef73-4030-8cfb-de9250e707b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26cd152e660e4e3d9996cabf7d18cdab'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:41.389612 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b9756b72-ef73-4030-8cfb-de9250e707b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26cd152e660e4e3d9996cabf7d18cdab'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:41.398237 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b9756b72-ef73-4030-8cfb-de9250e707b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26cd152e660e4e3d9996cabf7d18cdab'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:41.408518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 924/1830] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:41 2022] POST /identity/v3/projects => generated 407 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:36:41.410094 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0682c-e17e-4e2f-afc7-68d6fdde8bf9 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:41.412982 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b9756b72-ef73-4030-8cfb-de9250e707b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26cd152e660e4e3d9996cabf7d18cdab'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:41.413832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b9756b72-ef73-4030-8cfb-de9250e707b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26cd152e660e4e3d9996cabf7d18cdab'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:41.415528 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:41.422292 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 907/1831] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:41 2022] DELETE /identity/v3/users/26cd152e660e4e3d9996cabf7d18cdab => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:36:41.488852 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0682c-e17e-4e2f-afc7-68d6fdde8bf9 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:41.600165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3cceb-4696-420a-83f7-8d1b295fd63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:41.600813 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3cceb-4696-420a-83f7-8d1b295fd63e admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:41.601121 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3cceb-4696-420a-83f7-8d1b295fd63e admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:41.601402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3cceb-4696-420a-83f7-8d1b295fd63e admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:41.602084 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3cceb-4696-420a-83f7-8d1b295fd63e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:41.604029 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3cceb-4696-420a-83f7-8d1b295fd63e admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:41.638474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 925/1832] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:41 2022] POST /identity/v3/users => generated 304 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:41.642391 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9756b72-ef73-4030-8cfb-de9250e707b2 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:41.643857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b9756b72-ef73-4030-8cfb-de9250e707b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:41.671785 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9756b72-ef73-4030-8cfb-de9250e707b2 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:41.683717 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5260dcc-3f4d-4f80-b874-1657cf550671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:41.684405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5260dcc-3f4d-4f80-b874-1657cf550671 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:41.686771 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5260dcc-3f4d-4f80-b874-1657cf550671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:41.687070 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5260dcc-3f4d-4f80-b874-1657cf550671 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:41.690918 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5260dcc-3f4d-4f80-b874-1657cf550671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:41.702561 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5260dcc-3f4d-4f80-b874-1657cf550671 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:41.733291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 908/1833] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:41.734983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3cceb-4696-420a-83f7-8d1b295fd63e admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:41.764463 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3cceb-4696-420a-83f7-8d1b295fd63e admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:41.776581 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306cb15-6abf-4f2d-85fc-6497d3443dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:41.777664 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306cb15-6abf-4f2d-85fc-6497d3443dbf admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:41.777957 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306cb15-6abf-4f2d-85fc-6497d3443dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:41.778237 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306cb15-6abf-4f2d-85fc-6497d3443dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:41.780682 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306cb15-6abf-4f2d-85fc-6497d3443dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:41.784044 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306cb15-6abf-4f2d-85fc-6497d3443dbf admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:41.805256 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 926/1834] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:41.807445 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5260dcc-3f4d-4f80-b874-1657cf550671 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:41.832697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5260dcc-3f4d-4f80-b874-1657cf550671 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:41.850261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff44f8b-65ce-49c3-8fa1-b24716b021e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:41.851019 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff44f8b-65ce-49c3-8fa1-b24716b021e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:41.851307 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff44f8b-65ce-49c3-8fa1-b24716b021e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:41.852439 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff44f8b-65ce-49c3-8fa1-b24716b021e2 admin admin] PATH_INFO: `/v3/projects/6a344183ca8545908a2a0e6925f8c70d/users/03d6c3068089406a9fb57ad632d9e821/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:41.885259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff44f8b-65ce-49c3-8fa1-b24716b021e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a344183ca8545908a2a0e6925f8c70d, user_id=03d6c3068089406a9fb57ad632d9e821, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:41.899476 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff44f8b-65ce-49c3-8fa1-b24716b021e2 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:41.904362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306cb15-6abf-4f2d-85fc-6497d3443dbf admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:41.906361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7306cb15-6abf-4f2d-85fc-6497d3443dbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:41.929330 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306cb15-6abf-4f2d-85fc-6497d3443dbf admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:41.939809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 909/1835] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:41 2022] PUT /identity/v3/projects/6a344183ca8545908a2a0e6925f8c70d/users/03d6c3068089406a9fb57ad632d9e821/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:41.955288 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab703516-2463-4e2b-9ee9-7b7ab71ff919 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:41.955744 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab703516-2463-4e2b-9ee9-7b7ab71ff919 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:41.957053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab703516-2463-4e2b-9ee9-7b7ab71ff919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:42.037798 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-ab703516-2463-4e2b-9ee9-7b7ab71ff919 None None] MFA Rules not processed for user `03d6c3068089406a9fb57ad632d9e821`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:42.039074 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ab703516-2463-4e2b-9ee9-7b7ab71ff919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:42.059270 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:42.070386 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef6ace-ff73-488e-84e1-da70e56d9df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:42.071082 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef6ace-ff73-488e-84e1-da70e56d9df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:42.072432 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef6ace-ff73-488e-84e1-da70e56d9df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:42.072722 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef6ace-ff73-488e-84e1-da70e56d9df3 admin admin] PATH_INFO: `/v3/projects/ccd7282e500d4ddfa7fa0057fc1fab6c` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:42.073984 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef6ace-ff73-488e-84e1-da70e56d9df3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccd7282e500d4ddfa7fa0057fc1fab6c)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:42.077910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef6ace-ff73-488e-84e1-da70e56d9df3 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:42.154827 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:42.185361 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 910/1836] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:41 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:42.189213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab703516-2463-4e2b-9ee9-7b7ab71ff919 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:42.208117 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f5ef6ace-ff73-488e-84e1-da70e56d9df3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccd7282e500d4ddfa7fa0057fc1fab6c', 'request_id': 'req-f5ef6ace-ff73-488e-84e1-da70e56d9df3'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:42.217133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab703516-2463-4e2b-9ee9-7b7ab71ff919 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:42.221990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f5ef6ace-ff73-488e-84e1-da70e56d9df3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccd7282e500d4ddfa7fa0057fc1fab6c', 'request_id': 'req-f5ef6ace-ff73-488e-84e1-da70e56d9df3'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:42.228251 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f5ef6ace-ff73-488e-84e1-da70e56d9df3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccd7282e500d4ddfa7fa0057fc1fab6c', 'request_id': 'req-f5ef6ace-ff73-488e-84e1-da70e56d9df3'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:42.251315 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f5ef6ace-ff73-488e-84e1-da70e56d9df3 admin admin] The token cache is being invalidate because project ccd7282e500d4ddfa7fa0057fc1fab6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:42.251530 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f5ef6ace-ff73-488e-84e1-da70e56d9df3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:42.252194 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f5ef6ace-ff73-488e-84e1-da70e56d9df3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:42.253786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 927/1837] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:41 2022] DELETE /identity/v3/projects/ccd7282e500d4ddfa7fa0057fc1fab6c => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:42.258791 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111da11f-40af-4f7f-a781-5c725535314f None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:42.259082 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111da11f-40af-4f7f-a781-5c725535314f None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:42.259361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111da11f-40af-4f7f-a781-5c725535314f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:42.315023 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-111da11f-40af-4f7f-a781-5c725535314f None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:42.316178 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-111da11f-40af-4f7f-a781-5c725535314f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:42.354583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5356ecc-b374-460d-962a-ebfa2482e55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:42.355499 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5356ecc-b374-460d-962a-ebfa2482e55e admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:42.359229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5356ecc-b374-460d-962a-ebfa2482e55e admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:42.359754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5356ecc-b374-460d-962a-ebfa2482e55e admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:42.360684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5356ecc-b374-460d-962a-ebfa2482e55e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:42.366737 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5356ecc-b374-460d-962a-ebfa2482e55e admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:42.450021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 911/1838] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:42 2022] POST /identity/v3/projects => generated 407 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:42.454547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5356ecc-b374-460d-962a-ebfa2482e55e admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:42.456758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d5356ecc-b374-460d-962a-ebfa2482e55e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:42.484821 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 928/1839] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:42.493047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c839345-ff88-4ca9-95d5-72a9ca8bbf4f None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:42.493346 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c839345-ff88-4ca9-95d5-72a9ca8bbf4f None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:42.493630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c839345-ff88-4ca9-95d5-72a9ca8bbf4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:42.535514 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-3c839345-ff88-4ca9-95d5-72a9ca8bbf4f None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:42.541815 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-3c839345-ff88-4ca9-95d5-72a9ca8bbf4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:42.633294 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5356ecc-b374-460d-962a-ebfa2482e55e admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:42.641809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc0f9e-b6ec-4b5d-9a88-49706500af92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:42.642731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc0f9e-b6ec-4b5d-9a88-49706500af92 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:42.643220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc0f9e-b6ec-4b5d-9a88-49706500af92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:42.644168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc0f9e-b6ec-4b5d-9a88-49706500af92 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:42.646077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc0f9e-b6ec-4b5d-9a88-49706500af92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:42.648416 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc0f9e-b6ec-4b5d-9a88-49706500af92 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:42.663913 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 929/1840] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:36:42.664876 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c839345-ff88-4ca9-95d5-72a9ca8bbf4f None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:42.666552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-3c839345-ff88-4ca9-95d5-72a9ca8bbf4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:42.685954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c839345-ff88-4ca9-95d5-72a9ca8bbf4f None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:42.691944 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 912/1841] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:42 2022] POST /identity/v3/users => generated 310 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:42.693978 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc0f9e-b6ec-4b5d-9a88-49706500af92 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:42.702880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bdb0cf-50c4-49e5-a637-578b843b5302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:42.703645 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bdb0cf-50c4-49e5-a637-578b843b5302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:42.704064 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bdb0cf-50c4-49e5-a637-578b843b5302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:42.704387 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bdb0cf-50c4-49e5-a637-578b843b5302 admin admin] PATH_INFO: `/v3/users/08da8092815c42ac83ee3249d84a4941` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:42.705883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bdb0cf-50c4-49e5-a637-578b843b5302 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08da8092815c42ac83ee3249d84a4941)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:42.708031 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bdb0cf-50c4-49e5-a637-578b843b5302 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:42.709339 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc0f9e-b6ec-4b5d-9a88-49706500af92 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:42.717296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee799b1-04dc-4631-a303-e17dc6c541e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:42.718343 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee799b1-04dc-4631-a303-e17dc6c541e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:42.718777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee799b1-04dc-4631-a303-e17dc6c541e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:42.719195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee799b1-04dc-4631-a303-e17dc6c541e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:42.720674 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee799b1-04dc-4631-a303-e17dc6c541e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:36:42.721194 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee799b1-04dc-4631-a303-e17dc6c541e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:42.723458 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee799b1-04dc-4631-a303-e17dc6c541e9 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:42.743700 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 913/1842] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:36:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:36:42.748811 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee799b1-04dc-4631-a303-e17dc6c541e9 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:42.761279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee799b1-04dc-4631-a303-e17dc6c541e9 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:42.768562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70561c49-5c3f-4b7e-b8d3-c2b126922646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:42.769941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70561c49-5c3f-4b7e-b8d3-c2b126922646 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:42.770229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70561c49-5c3f-4b7e-b8d3-c2b126922646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:42.770509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70561c49-5c3f-4b7e-b8d3-c2b126922646 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:42.771366 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70561c49-5c3f-4b7e-b8d3-c2b126922646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:42.773118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70561c49-5c3f-4b7e-b8d3-c2b126922646 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:42.775976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-93bdb0cf-50c4-49e5-a637-578b843b5302 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08da8092815c42ac83ee3249d84a4941'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:42.790981 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 914/1843] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:42.790981 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70561c49-5c3f-4b7e-b8d3-c2b126922646 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:42.801645 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-93bdb0cf-50c4-49e5-a637-578b843b5302 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08da8092815c42ac83ee3249d84a4941'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:42.803528 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:42.805736 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70561c49-5c3f-4b7e-b8d3-c2b126922646 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:42.814248 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ec011-3a58-415a-ab56-c28e7ddc6f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:42.815025 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ec011-3a58-415a-ab56-c28e7ddc6f8b admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:42.815744 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ec011-3a58-415a-ab56-c28e7ddc6f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:42.815744 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ec011-3a58-415a-ab56-c28e7ddc6f8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:42.816417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ec011-3a58-415a-ab56-c28e7ddc6f8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:42.818512 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ec011-3a58-415a-ab56-c28e7ddc6f8b admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:42.838478 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:42.859893 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-93bdb0cf-50c4-49e5-a637-578b843b5302 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08da8092815c42ac83ee3249d84a4941'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:42.875027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-93bdb0cf-50c4-49e5-a637-578b843b5302 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08da8092815c42ac83ee3249d84a4941'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:42.876580 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-93bdb0cf-50c4-49e5-a637-578b843b5302 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08da8092815c42ac83ee3249d84a4941'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:42.892121 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 930/1844] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:42 2022] DELETE /identity/v3/users/08da8092815c42ac83ee3249d84a4941 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:42.892591 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bdb0cf-50c4-49e5-a637-578b843b5302 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:42.894159 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-93bdb0cf-50c4-49e5-a637-578b843b5302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:42.954321 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 915/1845] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:42 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:42.954321 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f1924-eaac-45f9-acf7-f5f8bf2a5ec2 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:42.954321 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f1924-eaac-45f9-acf7-f5f8bf2a5ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:42.955409 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f1924-eaac-45f9-acf7-f5f8bf2a5ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:42.983087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bdb0cf-50c4-49e5-a637-578b843b5302 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:42.995163 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-bd1f1924-eaac-45f9-acf7-f5f8bf2a5ec2 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:42.996419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-bd1f1924-eaac-45f9-acf7-f5f8bf2a5ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:43.050120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12b583-dd1c-4152-897f-8f8d45438b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:43.050932 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12b583-dd1c-4152-897f-8f8d45438b7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:43.051218 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12b583-dd1c-4152-897f-8f8d45438b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:43.051499 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12b583-dd1c-4152-897f-8f8d45438b7e admin admin] PATH_INFO: `/v3/projects/dd066fe4e9c2459c887a245e80c57694/users/2f8b140f9bd54211a41e05fd776dcfdf/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:43.070075 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12b583-dd1c-4152-897f-8f8d45438b7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd066fe4e9c2459c887a245e80c57694, user_id=2f8b140f9bd54211a41e05fd776dcfdf, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:43.072985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12b583-dd1c-4152-897f-8f8d45438b7e admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:43.097075 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 931/1846] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:42 2022] PUT /identity/v3/projects/dd066fe4e9c2459c887a245e80c57694/users/2f8b140f9bd54211a41e05fd776dcfdf/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:43.098142 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12b583-dd1c-4152-897f-8f8d45438b7e admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:43.103224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-4e12b583-dd1c-4152-897f-8f8d45438b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:43.181016 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 916/1847] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:43.182544 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f1924-eaac-45f9-acf7-f5f8bf2a5ec2 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:43.196596 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f1924-eaac-45f9-acf7-f5f8bf2a5ec2 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:43.215965 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12b583-dd1c-4152-897f-8f8d45438b7e admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:43.221713 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e4ddd-30e3-407d-bda1-931c63aa8be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:43.222438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e4ddd-30e3-407d-bda1-931c63aa8be4 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:43.222802 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e4ddd-30e3-407d-bda1-931c63aa8be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:43.223286 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e4ddd-30e3-407d-bda1-931c63aa8be4 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:43.224180 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e4ddd-30e3-407d-bda1-931c63aa8be4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:43.226512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e4ddd-30e3-407d-bda1-931c63aa8be4 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:43.237685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:43.260791 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 932/1848] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:43 2022] POST /identity/v3/users => generated 294 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:43.262454 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e4ddd-30e3-407d-bda1-931c63aa8be4 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:43.267319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-894e4ddd-30e3-407d-bda1-931c63aa8be4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:43.272628 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7ec17-f322-498c-9a63-abff8cc079a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:43.273306 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7ec17-f322-498c-9a63-abff8cc079a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:43.273590 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7ec17-f322-498c-9a63-abff8cc079a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:43.273979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7ec17-f322-498c-9a63-abff8cc079a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:43.274706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7ec17-f322-498c-9a63-abff8cc079a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:43.276501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7ec17-f322-498c-9a63-abff8cc079a7 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:43.288225 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 917/1849] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:43.289169 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7ec17-f322-498c-9a63-abff8cc079a7 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:43.300041 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7ec17-f322-498c-9a63-abff8cc079a7 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:43.306125 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91d8f3-6825-4597-bb2e-47b7c6a635c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:43.307036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91d8f3-6825-4597-bb2e-47b7c6a635c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:43.307328 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91d8f3-6825-4597-bb2e-47b7c6a635c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:43.307605 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91d8f3-6825-4597-bb2e-47b7c6a635c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:43.308300 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91d8f3-6825-4597-bb2e-47b7c6a635c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:43.311340 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91d8f3-6825-4597-bb2e-47b7c6a635c0 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:43.327254 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 918/1850] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:43.328777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91d8f3-6825-4597-bb2e-47b7c6a635c0 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:43.339911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91d8f3-6825-4597-bb2e-47b7c6a635c0 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:43.344927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f123de-89d8-478c-9218-2aaa3e2a654e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:43.345614 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f123de-89d8-478c-9218-2aaa3e2a654e admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:43.345899 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f123de-89d8-478c-9218-2aaa3e2a654e admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:43.346182 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f123de-89d8-478c-9218-2aaa3e2a654e admin admin] PATH_INFO: `/v3/projects/dd066fe4e9c2459c887a245e80c57694/users/2f8b140f9bd54211a41e05fd776dcfdf/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:43.349216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f123de-89d8-478c-9218-2aaa3e2a654e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd066fe4e9c2459c887a245e80c57694, user_id=2f8b140f9bd54211a41e05fd776dcfdf, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:43.349782 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e4ddd-30e3-407d-bda1-931c63aa8be4 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:43.352372 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f123de-89d8-478c-9218-2aaa3e2a654e admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:43.355540 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279895ca-ae52-4b2b-99a6-b79a25bddbdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:43.357143 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279895ca-ae52-4b2b-99a6-b79a25bddbdd service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:43.357681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279895ca-ae52-4b2b-99a6-b79a25bddbdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:43.358279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279895ca-ae52-4b2b-99a6-b79a25bddbdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:43.368530 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 919/1851] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:43 2022] PUT /identity/v3/projects/dd066fe4e9c2459c887a245e80c57694/users/2f8b140f9bd54211a41e05fd776dcfdf/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:43.369522 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f123de-89d8-478c-9218-2aaa3e2a654e admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:43.381462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279895ca-ae52-4b2b-99a6-b79a25bddbdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:43.383193 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279895ca-ae52-4b2b-99a6-b79a25bddbdd service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:43.399390 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 933/1852] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:43.401415 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dbf0d-8032-405e-be82-de46c1626752 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:43.401896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dbf0d-8032-405e-be82-de46c1626752 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:43.402254 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dbf0d-8032-405e-be82-de46c1626752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:43.448001 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-111dbf0d-8032-405e-be82-de46c1626752 None None] MFA Rules not processed for user `2f8b140f9bd54211a41e05fd776dcfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:43.449447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-111dbf0d-8032-405e-be82-de46c1626752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:43.459610 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:43.464903 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f123de-89d8-478c-9218-2aaa3e2a654e admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:43.472262 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae67b2-8928-4ad3-9205-ee709fb9d793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:43.473602 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae67b2-8928-4ad3-9205-ee709fb9d793 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:43.474223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae67b2-8928-4ad3-9205-ee709fb9d793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:43.474652 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae67b2-8928-4ad3-9205-ee709fb9d793 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:43.475660 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae67b2-8928-4ad3-9205-ee709fb9d793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:43.477858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae67b2-8928-4ad3-9205-ee709fb9d793 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:43.501520 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 920/1853] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:43.504234 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae67b2-8928-4ad3-9205-ee709fb9d793 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:43.518451 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae67b2-8928-4ad3-9205-ee709fb9d793 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:43.524819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18bced-9294-43cf-b8d0-5430e6b122e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:43.525927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18bced-9294-43cf-b8d0-5430e6b122e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:43.526310 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18bced-9294-43cf-b8d0-5430e6b122e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:43.526739 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18bced-9294-43cf-b8d0-5430e6b122e7 admin admin] PATH_INFO: `/v3/projects/ddf0a871bc524d699e5c885ab0cec3fb/users/b0957839746e4b2881eafc69cf8bc19e/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:43.567975 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18bced-9294-43cf-b8d0-5430e6b122e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddf0a871bc524d699e5c885ab0cec3fb, user_id=b0957839746e4b2881eafc69cf8bc19e, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:43.579830 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18bced-9294-43cf-b8d0-5430e6b122e7 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:43.642567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 921/1854] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:43 2022] PUT /identity/v3/projects/ddf0a871bc524d699e5c885ab0cec3fb/users/b0957839746e4b2881eafc69cf8bc19e/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:43.643848 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 934/1855] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:43 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:43.650415 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9eb275-b69a-4949-accf-bab326f89754 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:43.650712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9eb275-b69a-4949-accf-bab326f89754 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:43.651000 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9eb275-b69a-4949-accf-bab326f89754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:43.700043 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18bced-9294-43cf-b8d0-5430e6b122e7 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:43.703115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-aa18bced-9294-43cf-b8d0-5430e6b122e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:43.731112 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-7f9eb275-b69a-4949-accf-bab326f89754 None None] MFA Rules not processed for user `b0957839746e4b2881eafc69cf8bc19e`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:43.733259 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7f9eb275-b69a-4949-accf-bab326f89754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:43.825241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 935/1856] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:43.829728 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9eb275-b69a-4949-accf-bab326f89754 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:43.844029 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:43.847804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18bced-9294-43cf-b8d0-5430e6b122e7 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:43.861481 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175cb157-5bc4-462b-a27a-d307ac7e8dc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:43.862132 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175cb157-5bc4-462b-a27a-d307ac7e8dc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:43.862417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175cb157-5bc4-462b-a27a-d307ac7e8dc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:43.862691 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175cb157-5bc4-462b-a27a-d307ac7e8dc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:43.992567 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9eb275-b69a-4949-accf-bab326f89754 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:43.996946 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175cb157-5bc4-462b-a27a-d307ac7e8dc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:44.000046 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175cb157-5bc4-462b-a27a-d307ac7e8dc7 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:44.000620 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47f2d1-087d-45a8-8ecf-50e80ca21f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:44.001610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47f2d1-087d-45a8-8ecf-50e80ca21f1b admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:44.002010 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47f2d1-087d-45a8-8ecf-50e80ca21f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:44.002447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47f2d1-087d-45a8-8ecf-50e80ca21f1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:44.003252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47f2d1-087d-45a8-8ecf-50e80ca21f1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:44.007197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47f2d1-087d-45a8-8ecf-50e80ca21f1b admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:44.028086 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 922/1857] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:43 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:36:44.045688 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 936/1858] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:43 2022] POST /identity/v3/projects => generated 387 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:36:44.047348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175cb157-5bc4-462b-a27a-d307ac7e8dc7 service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:44.072132 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175cb157-5bc4-462b-a27a-d307ac7e8dc7 service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:44.173457 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a30ea4-09b6-4d0b-b854-9511d75aaa2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:44.174705 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a30ea4-09b6-4d0b-b854-9511d75aaa2d admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:44.175139 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a30ea4-09b6-4d0b-b854-9511d75aaa2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:44.175558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a30ea4-09b6-4d0b-b854-9511d75aaa2d admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:44.176623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a30ea4-09b6-4d0b-b854-9511d75aaa2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:44.179716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a30ea4-09b6-4d0b-b854-9511d75aaa2d admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:44.239190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 923/1859] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:44 2022] POST /identity/v3/users => generated 300 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:44.240468 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47f2d1-087d-45a8-8ecf-50e80ca21f1b admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:44.259726 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47f2d1-087d-45a8-8ecf-50e80ca21f1b admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:44.266419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcb160-de0d-4dce-80d0-d9eb9becc099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:44.267831 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcb160-de0d-4dce-80d0-d9eb9becc099 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:44.268132 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcb160-de0d-4dce-80d0-d9eb9becc099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:44.268535 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a30ea4-09b6-4d0b-b854-9511d75aaa2d admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:44.269971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcb160-de0d-4dce-80d0-d9eb9becc099 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:44.270359 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-31a30ea4-09b6-4d0b-b854-9511d75aaa2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:44.270487 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fcb160-de0d-4dce-80d0-d9eb9becc099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:44.274747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fcb160-de0d-4dce-80d0-d9eb9becc099 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:44.290737 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:44.296981 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 937/1860] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:44.298934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcb160-de0d-4dce-80d0-d9eb9becc099 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:44.313539 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcb160-de0d-4dce-80d0-d9eb9becc099 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:44.321539 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc21c6e-fe8a-464e-b522-257ddc424f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:44.322332 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc21c6e-fe8a-464e-b522-257ddc424f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:44.322714 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc21c6e-fe8a-464e-b522-257ddc424f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:44.323096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc21c6e-fe8a-464e-b522-257ddc424f1a admin admin] PATH_INFO: `/v3/projects/80622fc6b18f4acf827782dbbcd73e68/users/f4fffe3770e64840bf29527ca2b80cef/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:44.347810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc21c6e-fe8a-464e-b522-257ddc424f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=80622fc6b18f4acf827782dbbcd73e68, user_id=f4fffe3770e64840bf29527ca2b80cef, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:44.352843 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc21c6e-fe8a-464e-b522-257ddc424f1a admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:44.374975 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a30ea4-09b6-4d0b-b854-9511d75aaa2d admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:44.382435 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 938/1861] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:44 2022] PUT /identity/v3/projects/80622fc6b18f4acf827782dbbcd73e68/users/f4fffe3770e64840bf29527ca2b80cef/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:44.383845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc21c6e-fe8a-464e-b522-257ddc424f1a admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:44.438335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ea39e-feb1-4011-9caf-7e862c02b0e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:44.438980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ea39e-feb1-4011-9caf-7e862c02b0e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:44.439284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ea39e-feb1-4011-9caf-7e862c02b0e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:44.439563 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ea39e-feb1-4011-9caf-7e862c02b0e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:44.441764 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-7f3ea39e-feb1-4011-9caf-7e862c02b0e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:44.524199 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc21c6e-fe8a-464e-b522-257ddc424f1a admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:44.530899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf735d07-2916-4e81-a6a6-792b6e31469c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:44.534537 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf735d07-2916-4e81-a6a6-792b6e31469c admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:44.534537 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf735d07-2916-4e81-a6a6-792b6e31469c admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:44.534537 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf735d07-2916-4e81-a6a6-792b6e31469c admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:44.534537 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf735d07-2916-4e81-a6a6-792b6e31469c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:44.537881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf735d07-2916-4e81-a6a6-792b6e31469c admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:44.550474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ea39e-feb1-4011-9caf-7e862c02b0e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:44.552175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ea39e-feb1-4011-9caf-7e862c02b0e9 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:44.562562 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 939/1862] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:44.564625 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf735d07-2916-4e81-a6a6-792b6e31469c admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:44.566966 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 924/1863] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:44 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:36:44.609322 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf735d07-2916-4e81-a6a6-792b6e31469c admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:44.621762 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830a390-3eab-4d76-b7cd-1c7274dbfafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:44.623229 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830a390-3eab-4d76-b7cd-1c7274dbfafc admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:44.623693 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830a390-3eab-4d76-b7cd-1c7274dbfafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:44.624136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830a390-3eab-4d76-b7cd-1c7274dbfafc admin admin] PATH_INFO: `/v3/projects/80622fc6b18f4acf827782dbbcd73e68/users/f4fffe3770e64840bf29527ca2b80cef/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:44.628950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5830a390-3eab-4d76-b7cd-1c7274dbfafc admin admin] RBAC: Authorizing `identity:create_grant(project_id=80622fc6b18f4acf827782dbbcd73e68, user_id=f4fffe3770e64840bf29527ca2b80cef, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:44.635079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5830a390-3eab-4d76-b7cd-1c7274dbfafc admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:44.669900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 940/1864] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:44 2022] PUT /identity/v3/projects/80622fc6b18f4acf827782dbbcd73e68/users/f4fffe3770e64840bf29527ca2b80cef/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:44.681441 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315a9f8-c1ab-4ab3-83af-c66a0010a677 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:44.681869 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315a9f8-c1ab-4ab3-83af-c66a0010a677 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:44.682435 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315a9f8-c1ab-4ab3-83af-c66a0010a677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:44.733778 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-b315a9f8-c1ab-4ab3-83af-c66a0010a677 None None] MFA Rules not processed for user `f4fffe3770e64840bf29527ca2b80cef`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:44.735232 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b315a9f8-c1ab-4ab3-83af-c66a0010a677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:44.801596 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830a390-3eab-4d76-b7cd-1c7274dbfafc admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:44.803472 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-5830a390-3eab-4d76-b7cd-1c7274dbfafc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:44.915571 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 925/1865] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:44 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:44.930695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315a9f8-c1ab-4ab3-83af-c66a0010a677 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:44.943493 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:44.958089 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:45.018843 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830a390-3eab-4d76-b7cd-1c7274dbfafc admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:45.032567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315a9f8-c1ab-4ab3-83af-c66a0010a677 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:45.041877 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f742d46-eada-4a46-81f7-94d91335f2d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:45.042616 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f742d46-eada-4a46-81f7-94d91335f2d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:45.042803 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f742d46-eada-4a46-81f7-94d91335f2d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:45.043080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f742d46-eada-4a46-81f7-94d91335f2d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:45.047589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ab4ad-8962-4986-b982-f56de33319b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:45.048260 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ab4ad-8962-4986-b982-f56de33319b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:45.049270 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ab4ad-8962-4986-b982-f56de33319b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:45.049493 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ab4ad-8962-4986-b982-f56de33319b7 admin admin] PATH_INFO: `/v3/projects/a012ca76102a44f0bdb1f852d87a6558` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:45.052608 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ab4ad-8962-4986-b982-f56de33319b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a012ca76102a44f0bdb1f852d87a6558)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:45.055239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ab4ad-8962-4986-b982-f56de33319b7 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:45.060062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f742d46-eada-4a46-81f7-94d91335f2d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:45.061555 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f742d46-eada-4a46-81f7-94d91335f2d8 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:45.086431 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 926/1866] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:45.108969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f742d46-eada-4a46-81f7-94d91335f2d8 service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:45.125087 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f742d46-eada-4a46-81f7-94d91335f2d8 service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:45.131810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de61190-b26a-4478-a2d6-e8cc747bc9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:45.132666 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de61190-b26a-4478-a2d6-e8cc747bc9d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:45.133205 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de61190-b26a-4478-a2d6-e8cc747bc9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:45.133671 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de61190-b26a-4478-a2d6-e8cc747bc9d4 admin admin] PATH_INFO: `/v3/users/b0957839746e4b2881eafc69cf8bc19e` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:45.135013 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de61190-b26a-4478-a2d6-e8cc747bc9d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0957839746e4b2881eafc69cf8bc19e)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:45.137304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de61190-b26a-4478-a2d6-e8cc747bc9d4 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:45.177387 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-550ab4ad-8962-4986-b982-f56de33319b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a012ca76102a44f0bdb1f852d87a6558', 'request_id': 'req-550ab4ad-8962-4986-b982-f56de33319b7'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:45.189705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-550ab4ad-8962-4986-b982-f56de33319b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a012ca76102a44f0bdb1f852d87a6558', 'request_id': 'req-550ab4ad-8962-4986-b982-f56de33319b7'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:45.203702 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-550ab4ad-8962-4986-b982-f56de33319b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a012ca76102a44f0bdb1f852d87a6558', 'request_id': 'req-550ab4ad-8962-4986-b982-f56de33319b7'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:45.217496 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-550ab4ad-8962-4986-b982-f56de33319b7 admin admin] The token cache is being invalidate because project a012ca76102a44f0bdb1f852d87a6558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:45.218079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-550ab4ad-8962-4986-b982-f56de33319b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:45.219078 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-550ab4ad-8962-4986-b982-f56de33319b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:45.223624 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 941/1867] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:44 2022] DELETE /identity/v3/projects/a012ca76102a44f0bdb1f852d87a6558 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:45.230234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ab4ad-8962-4986-b982-f56de33319b7 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:45.235938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-550ab4ad-8962-4986-b982-f56de33319b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:45.269133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8de61190-b26a-4478-a2d6-e8cc747bc9d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0957839746e4b2881eafc69cf8bc19e'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:45.287235 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8de61190-b26a-4478-a2d6-e8cc747bc9d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0957839746e4b2881eafc69cf8bc19e'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:45.315961 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8de61190-b26a-4478-a2d6-e8cc747bc9d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0957839746e4b2881eafc69cf8bc19e'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:45.341450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8de61190-b26a-4478-a2d6-e8cc747bc9d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0957839746e4b2881eafc69cf8bc19e'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:45.342220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8de61190-b26a-4478-a2d6-e8cc747bc9d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0957839746e4b2881eafc69cf8bc19e'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:45.343095 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ab4ad-8962-4986-b982-f56de33319b7 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:45.352089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 927/1868] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:45 2022] DELETE /identity/v3/users/b0957839746e4b2881eafc69cf8bc19e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:45.356297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86f455-e944-45b3-ae97-c534c239bf5d None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:45.356595 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86f455-e944-45b3-ae97-c534c239bf5d None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:45.357116 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86f455-e944-45b3-ae97-c534c239bf5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:45.359793 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef35df7-fcb8-4ccf-9c49-8c5b625415c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:45.360604 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef35df7-fcb8-4ccf-9c49-8c5b625415c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:45.361004 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef35df7-fcb8-4ccf-9c49-8c5b625415c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:45.361369 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef35df7-fcb8-4ccf-9c49-8c5b625415c5 admin admin] PATH_INFO: `/v3/users/b5363093880a4c689d30dfcb61d37660` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:45.362693 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef35df7-fcb8-4ccf-9c49-8c5b625415c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5363093880a4c689d30dfcb61d37660)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:45.366156 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef35df7-fcb8-4ccf-9c49-8c5b625415c5 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:45.388692 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-9f86f455-e944-45b3-ae97-c534c239bf5d None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:45.390559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9f86f455-e944-45b3-ae97-c534c239bf5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:45.428582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-aef35df7-fcb8-4ccf-9c49-8c5b625415c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5363093880a4c689d30dfcb61d37660'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:45.442839 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-aef35df7-fcb8-4ccf-9c49-8c5b625415c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5363093880a4c689d30dfcb61d37660'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:45.466487 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-aef35df7-fcb8-4ccf-9c49-8c5b625415c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5363093880a4c689d30dfcb61d37660'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:45.469328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:45.476482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-aef35df7-fcb8-4ccf-9c49-8c5b625415c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5363093880a4c689d30dfcb61d37660'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:45.477398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-aef35df7-fcb8-4ccf-9c49-8c5b625415c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5363093880a4c689d30dfcb61d37660'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:45.483156 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 928/1869] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:45.485356 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 942/1870] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:45 2022] DELETE /identity/v3/users/b5363093880a4c689d30dfcb61d37660 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:45.495695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86f455-e944-45b3-ae97-c534c239bf5d None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:45.498474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9f86f455-e944-45b3-ae97-c534c239bf5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:45.526146 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:45.581228 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86f455-e944-45b3-ae97-c534c239bf5d None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:45.650967 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b6815-1177-4508-a18c-b28fda791154 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:45.652247 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b6815-1177-4508-a18c-b28fda791154 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:45.652807 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b6815-1177-4508-a18c-b28fda791154 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:45.653437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b6815-1177-4508-a18c-b28fda791154 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:45.655909 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-3f8b6815-1177-4508-a18c-b28fda791154 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:45.774888 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8b6815-1177-4508-a18c-b28fda791154 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:45.794136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8b6815-1177-4508-a18c-b28fda791154 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:45.821595 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 929/1871] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:46.129904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef35df7-fcb8-4ccf-9c49-8c5b625415c5 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:46.132138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-aef35df7-fcb8-4ccf-9c49-8c5b625415c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:46.159661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef35df7-fcb8-4ccf-9c49-8c5b625415c5 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:46.177499 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda9e1c-ebf4-4b36-8a39-10396b6f76a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:46.178187 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda9e1c-ebf4-4b36-8a39-10396b6f76a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:46.178594 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda9e1c-ebf4-4b36-8a39-10396b6f76a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:46.178878 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda9e1c-ebf4-4b36-8a39-10396b6f76a8 admin admin] PATH_INFO: `/v3/projects/d536cd0221ff4bf78bf0d7e65bc20e96` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:46.182699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda9e1c-ebf4-4b36-8a39-10396b6f76a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d536cd0221ff4bf78bf0d7e65bc20e96)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:46.188182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda9e1c-ebf4-4b36-8a39-10396b6f76a8 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:46.305262 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-dfda9e1c-ebf4-4b36-8a39-10396b6f76a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd536cd0221ff4bf78bf0d7e65bc20e96', 'request_id': 'req-dfda9e1c-ebf4-4b36-8a39-10396b6f76a8'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:46.318355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-dfda9e1c-ebf4-4b36-8a39-10396b6f76a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd536cd0221ff4bf78bf0d7e65bc20e96', 'request_id': 'req-dfda9e1c-ebf4-4b36-8a39-10396b6f76a8'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:46.342760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-dfda9e1c-ebf4-4b36-8a39-10396b6f76a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd536cd0221ff4bf78bf0d7e65bc20e96', 'request_id': 'req-dfda9e1c-ebf4-4b36-8a39-10396b6f76a8'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:46.397591 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-dfda9e1c-ebf4-4b36-8a39-10396b6f76a8 admin admin] The token cache is being invalidate because project d536cd0221ff4bf78bf0d7e65bc20e96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:46.398176 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-dfda9e1c-ebf4-4b36-8a39-10396b6f76a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:46.398884 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-dfda9e1c-ebf4-4b36-8a39-10396b6f76a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:46.409247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 943/1872] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:46 2022] DELETE /identity/v3/projects/d536cd0221ff4bf78bf0d7e65bc20e96 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:46.413521 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc48a3e-caf4-4e1a-8e77-78ee2cca26fc None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:46.413979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc48a3e-caf4-4e1a-8e77-78ee2cca26fc None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:46.414743 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc48a3e-caf4-4e1a-8e77-78ee2cca26fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:46.445323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-ffc48a3e-caf4-4e1a-8e77-78ee2cca26fc None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:46.446550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ffc48a3e-caf4-4e1a-8e77-78ee2cca26fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:46.505792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda9e1c-ebf4-4b36-8a39-10396b6f76a8 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:46.507316 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-dfda9e1c-ebf4-4b36-8a39-10396b6f76a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:46.539553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 930/1873] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:46.547601 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686b128-73cf-4276-8ebc-138cc56b9f03 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:46.548140 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686b128-73cf-4276-8ebc-138cc56b9f03 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:46.548746 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686b128-73cf-4276-8ebc-138cc56b9f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:46.585765 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-2686b128-73cf-4276-8ebc-138cc56b9f03 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:46.587429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-2686b128-73cf-4276-8ebc-138cc56b9f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:46.596050 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 931/1874] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:46.600381 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2686b128-73cf-4276-8ebc-138cc56b9f03 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:46.616430 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2686b128-73cf-4276-8ebc-138cc56b9f03 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:46.626065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132e15e-87b4-4193-8cdc-7c86e08a4b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:46.626930 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132e15e-87b4-4193-8cdc-7c86e08a4b17 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:46.627219 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132e15e-87b4-4193-8cdc-7c86e08a4b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:46.627598 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132e15e-87b4-4193-8cdc-7c86e08a4b17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:46.628262 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132e15e-87b4-4193-8cdc-7c86e08a4b17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:36:46.629452 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132e15e-87b4-4193-8cdc-7c86e08a4b17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:46.630321 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132e15e-87b4-4193-8cdc-7c86e08a4b17 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:46.637721 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda9e1c-ebf4-4b36-8a39-10396b6f76a8 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:46.648597 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 932/1875] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:36:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:36:46.650619 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132e15e-87b4-4193-8cdc-7c86e08a4b17 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:46.655730 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779e00f-158a-4c0e-a8f8-5dda988b863a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:46.656844 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779e00f-158a-4c0e-a8f8-5dda988b863a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:46.657983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779e00f-158a-4c0e-a8f8-5dda988b863a admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:46.658477 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779e00f-158a-4c0e-a8f8-5dda988b863a admin admin] PATH_INFO: `/v3/projects/ddf0a871bc524d699e5c885ab0cec3fb` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:46.663225 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779e00f-158a-4c0e-a8f8-5dda988b863a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddf0a871bc524d699e5c885ab0cec3fb)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:46.664731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132e15e-87b4-4193-8cdc-7c86e08a4b17 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:46.666810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779e00f-158a-4c0e-a8f8-5dda988b863a admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:46.672914 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1221544-cae1-46f4-8d1b-c3dfc1175902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:46.677617 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1221544-cae1-46f4-8d1b-c3dfc1175902 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:46.677896 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1221544-cae1-46f4-8d1b-c3dfc1175902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:46.678231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1221544-cae1-46f4-8d1b-c3dfc1175902 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:46.678944 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1221544-cae1-46f4-8d1b-c3dfc1175902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:46.681121 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1221544-cae1-46f4-8d1b-c3dfc1175902 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:46.687304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:46.735081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 933/1876] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:46 2022] POST /identity/v3/projects => generated 359 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:46.739053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1221544-cae1-46f4-8d1b-c3dfc1175902 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:46.774515 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-d779e00f-158a-4c0e-a8f8-5dda988b863a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddf0a871bc524d699e5c885ab0cec3fb', 'request_id': 'req-d779e00f-158a-4c0e-a8f8-5dda988b863a'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:46.786149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-d779e00f-158a-4c0e-a8f8-5dda988b863a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddf0a871bc524d699e5c885ab0cec3fb', 'request_id': 'req-d779e00f-158a-4c0e-a8f8-5dda988b863a'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:46.798996 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-d779e00f-158a-4c0e-a8f8-5dda988b863a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddf0a871bc524d699e5c885ab0cec3fb', 'request_id': 'req-d779e00f-158a-4c0e-a8f8-5dda988b863a'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:46.814830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-d779e00f-158a-4c0e-a8f8-5dda988b863a admin admin] The token cache is being invalidate because project ddf0a871bc524d699e5c885ab0cec3fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:46.815212 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-d779e00f-158a-4c0e-a8f8-5dda988b863a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:46.815968 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-d779e00f-158a-4c0e-a8f8-5dda988b863a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:46.819556 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 944/1877] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:46 2022] DELETE /identity/v3/projects/ddf0a871bc524d699e5c885ab0cec3fb => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:46.823290 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779e00f-158a-4c0e-a8f8-5dda988b863a admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:46.824802 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d779e00f-158a-4c0e-a8f8-5dda988b863a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:46.830191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1221544-cae1-46f4-8d1b-c3dfc1175902 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:46.832402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:46.900591 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f09ec2-79ca-4717-b1f7-8fa18f970d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:46.901272 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f09ec2-79ca-4717-b1f7-8fa18f970d9b admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:46.901557 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f09ec2-79ca-4717-b1f7-8fa18f970d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:46.901851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f09ec2-79ca-4717-b1f7-8fa18f970d9b admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:46.902570 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f09ec2-79ca-4717-b1f7-8fa18f970d9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:46.905093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f09ec2-79ca-4717-b1f7-8fa18f970d9b admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:46.908611 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779e00f-158a-4c0e-a8f8-5dda988b863a admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:46.913525 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0f8f3-12f8-43bb-a2ae-e5fd3e06d75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:46.914123 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0f8f3-12f8-43bb-a2ae-e5fd3e06d75b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:46.914413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0f8f3-12f8-43bb-a2ae-e5fd3e06d75b admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:46.914693 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0f8f3-12f8-43bb-a2ae-e5fd3e06d75b admin admin] PATH_INFO: `/v3/users/f4fffe3770e64840bf29527ca2b80cef` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:46.915924 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0f8f3-12f8-43bb-a2ae-e5fd3e06d75b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4fffe3770e64840bf29527ca2b80cef)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:46.918095 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0f8f3-12f8-43bb-a2ae-e5fd3e06d75b admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:46.936763 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 934/1878] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:46 2022] POST /identity/v3/users => generated 280 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:46.937676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f09ec2-79ca-4717-b1f7-8fa18f970d9b admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:46.939961 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-42f09ec2-79ca-4717-b1f7-8fa18f970d9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:46.954778 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f09ec2-79ca-4717-b1f7-8fa18f970d9b admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:46.972681 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5de85-585d-4ed5-8b9d-ffdec8e3a54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:46.975834 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5de85-585d-4ed5-8b9d-ffdec8e3a54e admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:46.976132 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5de85-585d-4ed5-8b9d-ffdec8e3a54e admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:46.977694 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5de85-585d-4ed5-8b9d-ffdec8e3a54e admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:46.979259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5de85-585d-4ed5-8b9d-ffdec8e3a54e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:46.982780 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5de85-585d-4ed5-8b9d-ffdec8e3a54e admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:46.998441 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 935/1879] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:47.000522 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5de85-585d-4ed5-8b9d-ffdec8e3a54e admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:47.006732 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-36c0f8f3-12f8-43bb-a2ae-e5fd3e06d75b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4fffe3770e64840bf29527ca2b80cef'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:47.011599 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5de85-585d-4ed5-8b9d-ffdec8e3a54e admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:47.019541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-36c0f8f3-12f8-43bb-a2ae-e5fd3e06d75b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4fffe3770e64840bf29527ca2b80cef'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:47.028136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6d628-596f-4ded-a4aa-a079a36b3b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:47.029867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6d628-596f-4ded-a4aa-a079a36b3b2f admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:47.030237 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6d628-596f-4ded-a4aa-a079a36b3b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:47.033475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6d628-596f-4ded-a4aa-a079a36b3b2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:47.034752 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6d628-596f-4ded-a4aa-a079a36b3b2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:47.037641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6d628-596f-4ded-a4aa-a079a36b3b2f admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:47.038310 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-36c0f8f3-12f8-43bb-a2ae-e5fd3e06d75b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4fffe3770e64840bf29527ca2b80cef'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:47.046105 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-36c0f8f3-12f8-43bb-a2ae-e5fd3e06d75b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4fffe3770e64840bf29527ca2b80cef'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:47.047268 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-36c0f8f3-12f8-43bb-a2ae-e5fd3e06d75b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4fffe3770e64840bf29527ca2b80cef'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:47.053674 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 945/1880] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:46 2022] DELETE /identity/v3/users/f4fffe3770e64840bf29527ca2b80cef => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:36:47.054263 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 936/1881] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:47.061194 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0f8f3-12f8-43bb-a2ae-e5fd3e06d75b admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:47.063065 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-36c0f8f3-12f8-43bb-a2ae-e5fd3e06d75b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:47.145013 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0f8f3-12f8-43bb-a2ae-e5fd3e06d75b admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:47.156409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b5488-af26-401c-b52c-9174c51bd04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:47.157224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b5488-af26-401c-b52c-9174c51bd04b admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:47.157684 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b5488-af26-401c-b52c-9174c51bd04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:47.158006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b5488-af26-401c-b52c-9174c51bd04b admin admin] PATH_INFO: `/v3/projects/eca572363c344754a823a35c93bc420e/users/170dc8ce14d4474d94e1902c54e4b15a/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:47.178931 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b5488-af26-401c-b52c-9174c51bd04b admin admin] RBAC: Authorizing `identity:create_grant(project_id=eca572363c344754a823a35c93bc420e, user_id=170dc8ce14d4474d94e1902c54e4b15a, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:47.181991 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b5488-af26-401c-b52c-9174c51bd04b admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:47.198623 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 946/1882] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:47 2022] PUT /identity/v3/projects/eca572363c344754a823a35c93bc420e/users/170dc8ce14d4474d94e1902c54e4b15a/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:47.209533 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03470fdc-348a-4c36-8463-df705230de9b None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:47.209934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03470fdc-348a-4c36-8463-df705230de9b None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:47.210117 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03470fdc-348a-4c36-8463-df705230de9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:47.256632 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-03470fdc-348a-4c36-8463-df705230de9b None None] MFA Rules not processed for user `170dc8ce14d4474d94e1902c54e4b15a`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:47.258176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-03470fdc-348a-4c36-8463-df705230de9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:47.339612 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:47.386595 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 937/1883] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:47 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:47.435273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b5488-af26-401c-b52c-9174c51bd04b admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:47.437307 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-079b5488-af26-401c-b52c-9174c51bd04b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:47.454319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:47.538002 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b5488-af26-401c-b52c-9174c51bd04b admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:47.542990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c40f7b-6974-49cf-9191-1dbbd2c3474e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:47.544643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c40f7b-6974-49cf-9191-1dbbd2c3474e service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:47.545146 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c40f7b-6974-49cf-9191-1dbbd2c3474e service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:47.545430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c40f7b-6974-49cf-9191-1dbbd2c3474e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:47.556598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c40f7b-6974-49cf-9191-1dbbd2c3474e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:47.558122 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c40f7b-6974-49cf-9191-1dbbd2c3474e service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:47.577249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 947/1884] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:47 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:36:47.597563 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03470fdc-348a-4c36-8463-df705230de9b None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:47.601512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c40f7b-6974-49cf-9191-1dbbd2c3474e service cinder] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:47.606976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-68c40f7b-6974-49cf-9191-1dbbd2c3474e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:47.693340 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03470fdc-348a-4c36-8463-df705230de9b None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:47.698791 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137fc4a-f410-4f34-806d-1bb1d7d2de95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:47.699443 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137fc4a-f410-4f34-806d-1bb1d7d2de95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:47.699727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137fc4a-f410-4f34-806d-1bb1d7d2de95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:47.700003 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137fc4a-f410-4f34-806d-1bb1d7d2de95 admin admin] PATH_INFO: `/v3/users/170dc8ce14d4474d94e1902c54e4b15a` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:47.701635 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137fc4a-f410-4f34-806d-1bb1d7d2de95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=170dc8ce14d4474d94e1902c54e4b15a)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:47.703600 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137fc4a-f410-4f34-806d-1bb1d7d2de95 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:47.722967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c40f7b-6974-49cf-9191-1dbbd2c3474e service cinder] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:47.728556 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff09ea8-4943-4505-8c5f-6ce1de848d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:47.729355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff09ea8-4943-4505-8c5f-6ce1de848d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:47.729708 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff09ea8-4943-4505-8c5f-6ce1de848d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:47.730060 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff09ea8-4943-4505-8c5f-6ce1de848d82 admin admin] PATH_INFO: `/v3/projects/80622fc6b18f4acf827782dbbcd73e68` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:47.731362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff09ea8-4943-4505-8c5f-6ce1de848d82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80622fc6b18f4acf827782dbbcd73e68)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:47.734214 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff09ea8-4943-4505-8c5f-6ce1de848d82 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:47.776555 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-2137fc4a-f410-4f34-806d-1bb1d7d2de95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '170dc8ce14d4474d94e1902c54e4b15a'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:47.786959 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-2137fc4a-f410-4f34-806d-1bb1d7d2de95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '170dc8ce14d4474d94e1902c54e4b15a'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:47.799381 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-2137fc4a-f410-4f34-806d-1bb1d7d2de95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '170dc8ce14d4474d94e1902c54e4b15a'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:47.818052 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-2137fc4a-f410-4f34-806d-1bb1d7d2de95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '170dc8ce14d4474d94e1902c54e4b15a'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:47.819519 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-2137fc4a-f410-4f34-806d-1bb1d7d2de95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '170dc8ce14d4474d94e1902c54e4b15a'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:47.831094 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1ff09ea8-4943-4505-8c5f-6ce1de848d82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80622fc6b18f4acf827782dbbcd73e68', 'request_id': 'req-1ff09ea8-4943-4505-8c5f-6ce1de848d82'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:47.843212 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1ff09ea8-4943-4505-8c5f-6ce1de848d82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80622fc6b18f4acf827782dbbcd73e68', 'request_id': 'req-1ff09ea8-4943-4505-8c5f-6ce1de848d82'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:47.844792 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 938/1885] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:47 2022] DELETE /identity/v3/users/170dc8ce14d4474d94e1902c54e4b15a => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:47.849299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834bc7d-6b50-41bc-8d79-4424c80a24a0 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:47.849597 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834bc7d-6b50-41bc-8d79-4424c80a24a0 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:47.849954 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834bc7d-6b50-41bc-8d79-4424c80a24a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:47.854756 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1ff09ea8-4943-4505-8c5f-6ce1de848d82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80622fc6b18f4acf827782dbbcd73e68', 'request_id': 'req-1ff09ea8-4943-4505-8c5f-6ce1de848d82'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:47.869884 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1ff09ea8-4943-4505-8c5f-6ce1de848d82 admin admin] The token cache is being invalidate because project 80622fc6b18f4acf827782dbbcd73e68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:47.870283 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1ff09ea8-4943-4505-8c5f-6ce1de848d82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:47.871212 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1ff09ea8-4943-4505-8c5f-6ce1de848d82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:47.873112 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 948/1886] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:47 2022] DELETE /identity/v3/projects/80622fc6b18f4acf827782dbbcd73e68 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:47.875751 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-e834bc7d-6b50-41bc-8d79-4424c80a24a0 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:47.877391 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-e834bc7d-6b50-41bc-8d79-4424c80a24a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:47.878173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f52d1-1944-4c09-b316-6675bf220ddf None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:47.878537 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f52d1-1944-4c09-b316-6675bf220ddf None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:47.878897 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f52d1-1944-4c09-b316-6675bf220ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:47.903012 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-e37f52d1-1944-4c09-b316-6675bf220ddf None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:47.905405 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e37f52d1-1944-4c09-b316-6675bf220ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:47.941104 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 939/1887] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:47.950236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:47.952727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834bc7d-6b50-41bc-8d79-4424c80a24a0 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:47.954354 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-e834bc7d-6b50-41bc-8d79-4424c80a24a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:47.968482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:47.993871 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 949/1888] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:36:48.001905 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ed081-8531-491a-ba66-f68c0e5fa0d2 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:48.002223 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ed081-8531-491a-ba66-f68c0e5fa0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:48.002545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ed081-8531-491a-ba66-f68c0e5fa0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:48.024262 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834bc7d-6b50-41bc-8d79-4424c80a24a0 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:48.027695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ede8f-542a-458b-8f93-fa6e8c981e20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:48.028317 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ede8f-542a-458b-8f93-fa6e8c981e20 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:48.029243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ede8f-542a-458b-8f93-fa6e8c981e20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:48.029518 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ede8f-542a-458b-8f93-fa6e8c981e20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:48.030950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-b64ed081-8531-491a-ba66-f68c0e5fa0d2 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:48.032172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b64ed081-8531-491a-ba66-f68c0e5fa0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:48.039114 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ede8f-542a-458b-8f93-fa6e8c981e20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:48.040554 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ede8f-542a-458b-8f93-fa6e8c981e20 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:48.042983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 950/1889] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:48.047799 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ed081-8531-491a-ba66-f68c0e5fa0d2 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:48.053222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 940/1890] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:48.057575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ed081-8531-491a-ba66-f68c0e5fa0d2 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:48.063255 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6352ff-b057-4752-92c9-4b570f8cc42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:48.064086 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6352ff-b057-4752-92c9-4b570f8cc42c admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:48.064416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6352ff-b057-4752-92c9-4b570f8cc42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:48.064695 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6352ff-b057-4752-92c9-4b570f8cc42c admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:48.065640 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6352ff-b057-4752-92c9-4b570f8cc42c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:36:48.065983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6352ff-b057-4752-92c9-4b570f8cc42c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:48.068465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6352ff-b057-4752-92c9-4b570f8cc42c admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:48.086002 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 951/1891] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:36:48.087505 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ede8f-542a-458b-8f93-fa6e8c981e20 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:48.100136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ede8f-542a-458b-8f93-fa6e8c981e20 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:48.106049 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f575a-abbf-464b-b730-f9d7dbd44034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:48.107052 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f575a-abbf-464b-b730-f9d7dbd44034 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:48.107300 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f575a-abbf-464b-b730-f9d7dbd44034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:48.107574 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f575a-abbf-464b-b730-f9d7dbd44034 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:48.108337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f575a-abbf-464b-b730-f9d7dbd44034 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:48.110267 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f575a-abbf-464b-b730-f9d7dbd44034 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:48.140798 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 941/1892] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:48 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:48.145543 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6352ff-b057-4752-92c9-4b570f8cc42c admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:48.157866 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6352ff-b057-4752-92c9-4b570f8cc42c admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:48.221789 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e72699c-ba1e-4ea6-8663-dc97cd824b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:48.222571 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72699c-ba1e-4ea6-8663-dc97cd824b51 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:48.222858 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72699c-ba1e-4ea6-8663-dc97cd824b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:48.223128 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72699c-ba1e-4ea6-8663-dc97cd824b51 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:48.223806 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e72699c-ba1e-4ea6-8663-dc97cd824b51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:48.225803 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e72699c-ba1e-4ea6-8663-dc97cd824b51 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:48.249078 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 952/1893] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:48 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:48.253214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f575a-abbf-464b-b730-f9d7dbd44034 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:48.264666 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f575a-abbf-464b-b730-f9d7dbd44034 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:48.269592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ebe8fe-fe9a-4d58-a9a9-92063b0e84cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:48.270216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ebe8fe-fe9a-4d58-a9a9-92063b0e84cd admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:48.270503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ebe8fe-fe9a-4d58-a9a9-92063b0e84cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:48.270990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ebe8fe-fe9a-4d58-a9a9-92063b0e84cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:48.271472 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ebe8fe-fe9a-4d58-a9a9-92063b0e84cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:48.273184 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ebe8fe-fe9a-4d58-a9a9-92063b0e84cd admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:48.284619 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 942/1894] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:36:48.289403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e72699c-ba1e-4ea6-8663-dc97cd824b51 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:48.300389 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e72699c-ba1e-4ea6-8663-dc97cd824b51 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:48.305273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca9dd3-a4bf-4f3c-ac82-b69d9523269d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:48.305900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca9dd3-a4bf-4f3c-ac82-b69d9523269d admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:48.306184 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca9dd3-a4bf-4f3c-ac82-b69d9523269d admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:48.306460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca9dd3-a4bf-4f3c-ac82-b69d9523269d admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:48.307157 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca9dd3-a4bf-4f3c-ac82-b69d9523269d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:48.308792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca9dd3-a4bf-4f3c-ac82-b69d9523269d admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:48.320454 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 953/1895] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:48.325056 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ebe8fe-fe9a-4d58-a9a9-92063b0e84cd admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:48.337747 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ebe8fe-fe9a-4d58-a9a9-92063b0e84cd admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:48.342721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd0720f-0cd1-4190-9108-2f74bc9112ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:48.343390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0720f-0cd1-4190-9108-2f74bc9112ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:48.343667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0720f-0cd1-4190-9108-2f74bc9112ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:48.343946 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0720f-0cd1-4190-9108-2f74bc9112ee admin admin] PATH_INFO: `/v3/projects/23cee0caab2748c58e7f4a12a62bb12d/users/e475f2128cc5461ebceec6464056950f/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:48.352050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca9dd3-a4bf-4f3c-ac82-b69d9523269d admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:48.354694 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d4ca9dd3-a4bf-4f3c-ac82-b69d9523269d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:48.360375 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd0720f-0cd1-4190-9108-2f74bc9112ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=23cee0caab2748c58e7f4a12a62bb12d, user_id=e475f2128cc5461ebceec6464056950f, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:48.363721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd0720f-0cd1-4190-9108-2f74bc9112ee admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:48.375887 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca9dd3-a4bf-4f3c-ac82-b69d9523269d admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:48.379871 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 943/1896] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:48 2022] PUT /identity/v3/projects/23cee0caab2748c58e7f4a12a62bb12d/users/e475f2128cc5461ebceec6464056950f/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:48.392805 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfdadf-df94-4ad8-9ad1-de5c945b2163 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:48.393374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfdadf-df94-4ad8-9ad1-de5c945b2163 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:48.393698 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfdadf-df94-4ad8-9ad1-de5c945b2163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:48.426999 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-fecfdadf-df94-4ad8-9ad1-de5c945b2163 None None] MFA Rules not processed for user `e475f2128cc5461ebceec6464056950f`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:48.428236 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-fecfdadf-df94-4ad8-9ad1-de5c945b2163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:48.449568 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d012e-3df0-4203-aa45-8461b13237ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:48.450231 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d012e-3df0-4203-aa45-8461b13237ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:48.450565 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d012e-3df0-4203-aa45-8461b13237ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:48.451398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d012e-3df0-4203-aa45-8461b13237ac admin admin] PATH_INFO: `/v3/projects/eca572363c344754a823a35c93bc420e` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:48.453587 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d012e-3df0-4203-aa45-8461b13237ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=eca572363c344754a823a35c93bc420e)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:48.456511 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d012e-3df0-4203-aa45-8461b13237ac admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:48.494679 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 944/1897] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:48.499629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfdadf-df94-4ad8-9ad1-de5c945b2163 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:48.501045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:48.508494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ba7d012e-3df0-4203-aa45-8461b13237ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eca572363c344754a823a35c93bc420e', 'request_id': 'req-ba7d012e-3df0-4203-aa45-8461b13237ac'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:48.512526 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfdadf-df94-4ad8-9ad1-de5c945b2163 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:48.517740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ba7d012e-3df0-4203-aa45-8461b13237ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eca572363c344754a823a35c93bc420e', 'request_id': 'req-ba7d012e-3df0-4203-aa45-8461b13237ac'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:48.524873 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ba7d012e-3df0-4203-aa45-8461b13237ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eca572363c344754a823a35c93bc420e', 'request_id': 'req-ba7d012e-3df0-4203-aa45-8461b13237ac'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:48.538326 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ba7d012e-3df0-4203-aa45-8461b13237ac admin admin] The token cache is being invalidate because project eca572363c344754a823a35c93bc420e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:48.538622 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ba7d012e-3df0-4203-aa45-8461b13237ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:48.539298 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ba7d012e-3df0-4203-aa45-8461b13237ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:48.541498 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 954/1898] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:48 2022] DELETE /identity/v3/projects/eca572363c344754a823a35c93bc420e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:36:48.546575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f6600-0b0c-4ae9-8d8a-d79164f90f6d None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:48.546866 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f6600-0b0c-4ae9-8d8a-d79164f90f6d None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:48.547143 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f6600-0b0c-4ae9-8d8a-d79164f90f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:48.562113 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:48.568733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-a11f6600-0b0c-4ae9-8d8a-d79164f90f6d None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:48.570301 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-a11f6600-0b0c-4ae9-8d8a-d79164f90f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:48.584848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5405db-4588-476a-816b-7becd3dbf181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:48.585850 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5405db-4588-476a-816b-7becd3dbf181 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:48.586143 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5405db-4588-476a-816b-7becd3dbf181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:48.586467 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5405db-4588-476a-816b-7becd3dbf181 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:48.587281 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5405db-4588-476a-816b-7becd3dbf181 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:48.589688 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5405db-4588-476a-816b-7becd3dbf181 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:48.632458 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 945/1899] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:48 2022] POST /identity/v3/projects => generated 381 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:36:48.633654 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5405db-4588-476a-816b-7becd3dbf181 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:48.635801 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-3c5405db-4588-476a-816b-7becd3dbf181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:48.640453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 955/1900] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:36:48.649347 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5b1eb-2b86-44c2-b1e4-ec1bc77e9e6e None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:48.649633 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5b1eb-2b86-44c2-b1e4-ec1bc77e9e6e None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:48.650442 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5b1eb-2b86-44c2-b1e4-ec1bc77e9e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:48.673646 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-6bd5b1eb-2b86-44c2-b1e4-ec1bc77e9e6e None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:48.674894 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-6bd5b1eb-2b86-44c2-b1e4-ec1bc77e9e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:48.718195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5405db-4588-476a-816b-7becd3dbf181 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:48.723013 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2c2ad-c784-4c4f-a4d0-6ce35190fe8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:48.723841 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2c2ad-c784-4c4f-a4d0-6ce35190fe8f admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:48.724126 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2c2ad-c784-4c4f-a4d0-6ce35190fe8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:48.724403 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2c2ad-c784-4c4f-a4d0-6ce35190fe8f admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:48.725101 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2c2ad-c784-4c4f-a4d0-6ce35190fe8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:48.727032 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2c2ad-c784-4c4f-a4d0-6ce35190fe8f admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:48.743635 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 956/1901] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:36:48.747923 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5b1eb-2b86-44c2-b1e4-ec1bc77e9e6e None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:48.750586 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 946/1902] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:48 2022] POST /identity/v3/users => generated 297 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:48.754271 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2c2ad-c784-4c4f-a4d0-6ce35190fe8f admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:48.759589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5b1eb-2b86-44c2-b1e4-ec1bc77e9e6e None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:48.764786 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2c2ad-c784-4c4f-a4d0-6ce35190fe8f admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:48.765097 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d73bab-7e9a-4414-88f2-a5e2e339174f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:48.765739 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d73bab-7e9a-4414-88f2-a5e2e339174f admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:48.766041 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d73bab-7e9a-4414-88f2-a5e2e339174f admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:48.766330 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d73bab-7e9a-4414-88f2-a5e2e339174f admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:48.767070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d73bab-7e9a-4414-88f2-a5e2e339174f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:36:48.767455 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d73bab-7e9a-4414-88f2-a5e2e339174f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:48.769089 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d73bab-7e9a-4414-88f2-a5e2e339174f admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:48.770169 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705ae92-397d-4ef1-b459-e50c03c4df41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:48.770897 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705ae92-397d-4ef1-b459-e50c03c4df41 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:48.771184 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705ae92-397d-4ef1-b459-e50c03c4df41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:48.771809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705ae92-397d-4ef1-b459-e50c03c4df41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:48.772149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705ae92-397d-4ef1-b459-e50c03c4df41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:48.773779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705ae92-397d-4ef1-b459-e50c03c4df41 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:48.781425 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 957/1903] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:36:48.785740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d73bab-7e9a-4414-88f2-a5e2e339174f admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:48.786162 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 947/1904] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:36:48.789929 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705ae92-397d-4ef1-b459-e50c03c4df41 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:48.795985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d73bab-7e9a-4414-88f2-a5e2e339174f admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:48.799673 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705ae92-397d-4ef1-b459-e50c03c4df41 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:48.800861 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caabb409-f0f6-4fc4-bf04-57f8641cc142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:48.801508 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabb409-f0f6-4fc4-bf04-57f8641cc142 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:48.802339 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabb409-f0f6-4fc4-bf04-57f8641cc142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:48.802625 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabb409-f0f6-4fc4-bf04-57f8641cc142 admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:48.803279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caabb409-f0f6-4fc4-bf04-57f8641cc142 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:48.805064 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca99516-0bcd-4d07-bb45-d0adf9c044de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:48.805387 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caabb409-f0f6-4fc4-bf04-57f8641cc142 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:48.805724 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca99516-0bcd-4d07-bb45-d0adf9c044de admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:48.806007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca99516-0bcd-4d07-bb45-d0adf9c044de admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:48.806317 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca99516-0bcd-4d07-bb45-d0adf9c044de admin admin] PATH_INFO: `/v3/projects/3a82a22030834390b397559317ef19ac/users/8c9c20bed80740f396dc9609d0fec026/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:48.822151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca99516-0bcd-4d07-bb45-d0adf9c044de admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a82a22030834390b397559317ef19ac, user_id=8c9c20bed80740f396dc9609d0fec026, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:48.826240 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca99516-0bcd-4d07-bb45-d0adf9c044de admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:48.836502 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 958/1905] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:48 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:36:48.837524 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caabb409-f0f6-4fc4-bf04-57f8641cc142 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:48.849656 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 948/1906] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:48 2022] PUT /identity/v3/projects/3a82a22030834390b397559317ef19ac/users/8c9c20bed80740f396dc9609d0fec026/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:36:48.854104 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca99516-0bcd-4d07-bb45-d0adf9c044de admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:48.884632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:48.910479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caabb409-f0f6-4fc4-bf04-57f8641cc142 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:48.921278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca99516-0bcd-4d07-bb45-d0adf9c044de admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:48.925901 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854de99-f9d6-476e-8974-6ee359172080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:48.926528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854de99-f9d6-476e-8974-6ee359172080 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:48.926809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854de99-f9d6-476e-8974-6ee359172080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:48.927084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854de99-f9d6-476e-8974-6ee359172080 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:48.927774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854de99-f9d6-476e-8974-6ee359172080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:48.929477 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854de99-f9d6-476e-8974-6ee359172080 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:48.939437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 949/1907] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:36:48.943539 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854de99-f9d6-476e-8974-6ee359172080 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:48.954152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854de99-f9d6-476e-8974-6ee359172080 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:48.960196 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ffa64-1f4b-458a-ba61-279cf1688900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:48.960973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ffa64-1f4b-458a-ba61-279cf1688900 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:48.961195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ffa64-1f4b-458a-ba61-279cf1688900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:48.961475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ffa64-1f4b-458a-ba61-279cf1688900 admin admin] PATH_INFO: `/v3/projects/3a82a22030834390b397559317ef19ac/users/8c9c20bed80740f396dc9609d0fec026/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:48.963738 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ffa64-1f4b-458a-ba61-279cf1688900 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a82a22030834390b397559317ef19ac, user_id=8c9c20bed80740f396dc9609d0fec026, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:48.967058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ffa64-1f4b-458a-ba61-279cf1688900 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:48.968572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33945a6-ccf5-4b54-8fca-8e66d8989cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:48.969169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33945a6-ccf5-4b54-8fca-8e66d8989cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:48.969477 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33945a6-ccf5-4b54-8fca-8e66d8989cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:48.969783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33945a6-ccf5-4b54-8fca-8e66d8989cc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:48.970697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33945a6-ccf5-4b54-8fca-8e66d8989cc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:48.973685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33945a6-ccf5-4b54-8fca-8e66d8989cc9 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:48.981012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 950/1908] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:48 2022] PUT /identity/v3/projects/3a82a22030834390b397559317ef19ac/users/8c9c20bed80740f396dc9609d0fec026/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:36:48.991608 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125075c-b4e3-491a-a94f-40f3271d10ff None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:48.991906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125075c-b4e3-491a-a94f-40f3271d10ff None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:48.992188 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125075c-b4e3-491a-a94f-40f3271d10ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:49.000629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 959/1909] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:48 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:49.001706 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33945a6-ccf5-4b54-8fca-8e66d8989cc9 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:49.008342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:49.014063 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33945a6-ccf5-4b54-8fca-8e66d8989cc9 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:49.023566 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-9125075c-b4e3-491a-a94f-40f3271d10ff None None] MFA Rules not processed for user `8c9c20bed80740f396dc9609d0fec026`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:49.024774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9125075c-b4e3-491a-a94f-40f3271d10ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:49.079700 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9746c01-3d99-490f-8f10-d4b81f35e6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:49.080342 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9746c01-3d99-490f-8f10-d4b81f35e6ee admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:49.080632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9746c01-3d99-490f-8f10-d4b81f35e6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:49.080928 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9746c01-3d99-490f-8f10-d4b81f35e6ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:49.081952 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9746c01-3d99-490f-8f10-d4b81f35e6ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:49.083554 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9746c01-3d99-490f-8f10-d4b81f35e6ee admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:49.095834 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 960/1910] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:49.099260 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9746c01-3d99-490f-8f10-d4b81f35e6ee admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:49.112038 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 951/1911] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:36:49.114036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9746c01-3d99-490f-8f10-d4b81f35e6ee admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:49.125185 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125075c-b4e3-491a-a94f-40f3271d10ff None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:49.126285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a2744-2067-4b96-82c4-b060ea372c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:49.126946 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a2744-2067-4b96-82c4-b060ea372c01 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:49.127232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a2744-2067-4b96-82c4-b060ea372c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:49.127402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9125075c-b4e3-491a-a94f-40f3271d10ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:49.127518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a2744-2067-4b96-82c4-b060ea372c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:49.128208 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a2744-2067-4b96-82c4-b060ea372c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:49.129851 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a2744-2067-4b96-82c4-b060ea372c01 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:49.149053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 961/1912] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:49.153097 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a2744-2067-4b96-82c4-b060ea372c01 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:49.166172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a2744-2067-4b96-82c4-b060ea372c01 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:49.171518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe176b85-8c40-4a99-945c-4ebd5015de2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:49.172208 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe176b85-8c40-4a99-945c-4ebd5015de2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:49.172489 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe176b85-8c40-4a99-945c-4ebd5015de2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:49.172766 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe176b85-8c40-4a99-945c-4ebd5015de2a admin admin] PATH_INFO: `/v3/projects/fef3e0b606114adfbb6ff0c7e5648bea/users/4fe9ea35543546afab739c0cfa3389fc/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:49.192665 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe176b85-8c40-4a99-945c-4ebd5015de2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fef3e0b606114adfbb6ff0c7e5648bea, user_id=4fe9ea35543546afab739c0cfa3389fc, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:49.195575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe176b85-8c40-4a99-945c-4ebd5015de2a admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:49.212974 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 962/1913] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:49 2022] PUT /identity/v3/projects/fef3e0b606114adfbb6ff0c7e5648bea/users/4fe9ea35543546afab739c0cfa3389fc/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:36:49.216846 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02819457-7854-4a94-8c99-7c97bffd4743 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:49.217249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02819457-7854-4a94-8c99-7c97bffd4743 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:49.217596 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02819457-7854-4a94-8c99-7c97bffd4743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:49.229796 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125075c-b4e3-491a-a94f-40f3271d10ff None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:49.238178 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-02819457-7854-4a94-8c99-7c97bffd4743 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:49.239387 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-02819457-7854-4a94-8c99-7c97bffd4743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:49.272932 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:49.299253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503adea6-3f85-49b4-983e-c83b551259b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:49.299581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503adea6-3f85-49b4-983e-c83b551259b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:49.300005 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503adea6-3f85-49b4-983e-c83b551259b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:49.300381 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503adea6-3f85-49b4-983e-c83b551259b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:49.313030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 963/1914] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:36:49.322919 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02819457-7854-4a94-8c99-7c97bffd4743 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:49.324554 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-02819457-7854-4a94-8c99-7c97bffd4743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:49.367458 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503adea6-3f85-49b4-983e-c83b551259b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:49.368931 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503adea6-3f85-49b4-983e-c83b551259b7 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:49.373042 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:49.380521 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 952/1915] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:49 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:36:49.388945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02819457-7854-4a94-8c99-7c97bffd4743 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:49.392589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c42d6-ed59-4e20-b926-5257a4b301c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:49.393240 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c42d6-ed59-4e20-b926-5257a4b301c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:49.393521 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c42d6-ed59-4e20-b926-5257a4b301c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:49.393795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c42d6-ed59-4e20-b926-5257a4b301c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:49.404373 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c42d6-ed59-4e20-b926-5257a4b301c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:49.406517 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c42d6-ed59-4e20-b926-5257a4b301c8 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:49.416791 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 964/1916] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:36:49.489070 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503adea6-3f85-49b4-983e-c83b551259b7 service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:49.498796 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503adea6-3f85-49b4-983e-c83b551259b7 service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:49.503513 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d4dac-7915-4f98-af3b-b128cd2751cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:49.504192 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d4dac-7915-4f98-af3b-b128cd2751cd service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:49.504611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d4dac-7915-4f98-af3b-b128cd2751cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:49.504911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d4dac-7915-4f98-af3b-b128cd2751cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:49.507027 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-bb5d4dac-7915-4f98-af3b-b128cd2751cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:49.573067 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d4dac-7915-4f98-af3b-b128cd2751cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:49.574534 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d4dac-7915-4f98-af3b-b128cd2751cd service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:49.585618 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 953/1917] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:50.259405 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab11ff-a5fb-46ce-bc05-5bac4b29a6ce None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:50.260089 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab11ff-a5fb-46ce-bc05-5bac4b29a6ce None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:50.260089 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab11ff-a5fb-46ce-bc05-5bac4b29a6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:50.293025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-b5ab11ff-a5fb-46ce-bc05-5bac4b29a6ce None None] MFA Rules not processed for user `4fe9ea35543546afab739c0cfa3389fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:50.294776 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b5ab11ff-a5fb-46ce-bc05-5bac4b29a6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:50.378379 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 965/1918] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:50.396327 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d4dac-7915-4f98-af3b-b128cd2751cd service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:50.398046 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-bb5d4dac-7915-4f98-af3b-b128cd2751cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:50.413453 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d4dac-7915-4f98-af3b-b128cd2751cd service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:50.422050 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdfe01-2384-4f56-b237-0c890f6add9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:50.422818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdfe01-2384-4f56-b237-0c890f6add9e service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:50.423211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdfe01-2384-4f56-b237-0c890f6add9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:50.423571 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdfe01-2384-4f56-b237-0c890f6add9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:50.438889 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdfe01-2384-4f56-b237-0c890f6add9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:50.440392 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdfe01-2384-4f56-b237-0c890f6add9e service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:50.450051 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 954/1919] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:51.824176 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5482c-aa49-4f2d-8061-e0d6365f77d2 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:51.824769 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5482c-aa49-4f2d-8061-e0d6365f77d2 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:51.824769 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5482c-aa49-4f2d-8061-e0d6365f77d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:51.846302 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-43b5482c-aa49-4f2d-8061-e0d6365f77d2 None None] MFA Rules not processed for user `4fe9ea35543546afab739c0cfa3389fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:51.847781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-43b5482c-aa49-4f2d-8061-e0d6365f77d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:51.856254 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 966/1920] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:52.450024 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdfe01-2384-4f56-b237-0c890f6add9e service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:52.478463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdfe01-2384-4f56-b237-0c890f6add9e service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:52.501403 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4bc780-5811-46f0-b6d7-319583109322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:52.502328 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bc780-5811-46f0-b6d7-319583109322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:52.502627 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bc780-5811-46f0-b6d7-319583109322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:52.503018 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bc780-5811-46f0-b6d7-319583109322 admin admin] PATH_INFO: `/v3/users/e475f2128cc5461ebceec6464056950f` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:52.509710 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4bc780-5811-46f0-b6d7-319583109322 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e475f2128cc5461ebceec6464056950f)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:52.512767 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4bc780-5811-46f0-b6d7-319583109322 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:52.618437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-de4bc780-5811-46f0-b6d7-319583109322 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e475f2128cc5461ebceec6464056950f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:52.625384 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-de4bc780-5811-46f0-b6d7-319583109322 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e475f2128cc5461ebceec6464056950f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:52.645441 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-de4bc780-5811-46f0-b6d7-319583109322 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e475f2128cc5461ebceec6464056950f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:52.671051 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-de4bc780-5811-46f0-b6d7-319583109322 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e475f2128cc5461ebceec6464056950f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:52.673083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-de4bc780-5811-46f0-b6d7-319583109322 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e475f2128cc5461ebceec6464056950f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:52.680344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 955/1921] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:52 2022] DELETE /identity/v3/users/e475f2128cc5461ebceec6464056950f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:52.685519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ee10e-c35b-4958-ad5a-ea0873625e7d None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:52.686003 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ee10e-c35b-4958-ad5a-ea0873625e7d None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:52.686290 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ee10e-c35b-4958-ad5a-ea0873625e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:52.753417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-017ee10e-c35b-4958-ad5a-ea0873625e7d None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:52.755183 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-017ee10e-c35b-4958-ad5a-ea0873625e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:52.850869 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:52.863902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 967/1922] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:36:52.878906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4bc780-5811-46f0-b6d7-319583109322 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:52.880963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-de4bc780-5811-46f0-b6d7-319583109322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:52.990499 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4bc780-5811-46f0-b6d7-319583109322 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:53.002803 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44045c-16cc-4eb2-a972-4a7f1f635824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:53.003647 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44045c-16cc-4eb2-a972-4a7f1f635824 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:53.004686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44045c-16cc-4eb2-a972-4a7f1f635824 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:53.005337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44045c-16cc-4eb2-a972-4a7f1f635824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:53.022113 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44045c-16cc-4eb2-a972-4a7f1f635824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:53.023707 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44045c-16cc-4eb2-a972-4a7f1f635824 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:53.028003 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:53.036125 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 956/1923] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:53.559264 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ee10e-c35b-4958-ad5a-ea0873625e7d None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:53.561371 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-017ee10e-c35b-4958-ad5a-ea0873625e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:53.574442 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ee10e-c35b-4958-ad5a-ea0873625e7d None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:53.579904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf9612-627f-4508-9799-2f425a33a7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:53.580560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf9612-627f-4508-9799-2f425a33a7c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:53.580850 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf9612-627f-4508-9799-2f425a33a7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:53.581167 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf9612-627f-4508-9799-2f425a33a7c2 admin admin] PATH_INFO: `/v3/projects/23cee0caab2748c58e7f4a12a62bb12d` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:53.582351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf9612-627f-4508-9799-2f425a33a7c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23cee0caab2748c58e7f4a12a62bb12d)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:53.584586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf9612-627f-4508-9799-2f425a33a7c2 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:53.632644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-8abf9612-627f-4508-9799-2f425a33a7c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23cee0caab2748c58e7f4a12a62bb12d', 'request_id': 'req-8abf9612-627f-4508-9799-2f425a33a7c2'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:53.640382 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-8abf9612-627f-4508-9799-2f425a33a7c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23cee0caab2748c58e7f4a12a62bb12d', 'request_id': 'req-8abf9612-627f-4508-9799-2f425a33a7c2'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:53.648492 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-8abf9612-627f-4508-9799-2f425a33a7c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23cee0caab2748c58e7f4a12a62bb12d', 'request_id': 'req-8abf9612-627f-4508-9799-2f425a33a7c2'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:53.660849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-8abf9612-627f-4508-9799-2f425a33a7c2 admin admin] The token cache is being invalidate because project 23cee0caab2748c58e7f4a12a62bb12d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:53.661108 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-8abf9612-627f-4508-9799-2f425a33a7c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:53.661813 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-8abf9612-627f-4508-9799-2f425a33a7c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:53.663367 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 968/1924] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:53 2022] DELETE /identity/v3/projects/23cee0caab2748c58e7f4a12a62bb12d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:53.667431 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44045c-16cc-4eb2-a972-4a7f1f635824 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:53.678184 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44045c-16cc-4eb2-a972-4a7f1f635824 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:53.734066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df85bfe-6fa9-4506-95d5-15972abfb190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:53.734713 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df85bfe-6fa9-4506-95d5-15972abfb190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:53.734998 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df85bfe-6fa9-4506-95d5-15972abfb190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:53.735276 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df85bfe-6fa9-4506-95d5-15972abfb190 admin admin] PATH_INFO: `/v3/users/8c9c20bed80740f396dc9609d0fec026` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:53.736445 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df85bfe-6fa9-4506-95d5-15972abfb190 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c9c20bed80740f396dc9609d0fec026)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:53.738404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df85bfe-6fa9-4506-95d5-15972abfb190 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:53.798422 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-5df85bfe-6fa9-4506-95d5-15972abfb190 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c9c20bed80740f396dc9609d0fec026'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:53.808258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-5df85bfe-6fa9-4506-95d5-15972abfb190 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c9c20bed80740f396dc9609d0fec026'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:53.818970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-5df85bfe-6fa9-4506-95d5-15972abfb190 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c9c20bed80740f396dc9609d0fec026'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:53.837045 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-5df85bfe-6fa9-4506-95d5-15972abfb190 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c9c20bed80740f396dc9609d0fec026'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:53.837775 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-5df85bfe-6fa9-4506-95d5-15972abfb190 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c9c20bed80740f396dc9609d0fec026'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:53.847694 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 957/1925] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:53 2022] DELETE /identity/v3/users/8c9c20bed80740f396dc9609d0fec026 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:53.946196 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf9612-627f-4508-9799-2f425a33a7c2 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:53.948818 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-8abf9612-627f-4508-9799-2f425a33a7c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:53.968195 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf9612-627f-4508-9799-2f425a33a7c2 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:54.039138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682599b-2b8a-42f5-9bb2-ec1569553b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:54.039800 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682599b-2b8a-42f5-9bb2-ec1569553b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:54.040090 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682599b-2b8a-42f5-9bb2-ec1569553b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:54.040374 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682599b-2b8a-42f5-9bb2-ec1569553b48 admin admin] PATH_INFO: `/v3/users/03d6c3068089406a9fb57ad632d9e821` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:54.041670 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682599b-2b8a-42f5-9bb2-ec1569553b48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03d6c3068089406a9fb57ad632d9e821)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:54.043815 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682599b-2b8a-42f5-9bb2-ec1569553b48 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:54.098580 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-b682599b-2b8a-42f5-9bb2-ec1569553b48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03d6c3068089406a9fb57ad632d9e821'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:54.108025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-b682599b-2b8a-42f5-9bb2-ec1569553b48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03d6c3068089406a9fb57ad632d9e821'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:54.120317 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-b682599b-2b8a-42f5-9bb2-ec1569553b48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03d6c3068089406a9fb57ad632d9e821'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:54.127663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-b682599b-2b8a-42f5-9bb2-ec1569553b48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03d6c3068089406a9fb57ad632d9e821'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:54.128458 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-b682599b-2b8a-42f5-9bb2-ec1569553b48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03d6c3068089406a9fb57ad632d9e821'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:54.134806 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 969/1926] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:53 2022] DELETE /identity/v3/users/03d6c3068089406a9fb57ad632d9e821 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:54.140685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4e4f7-0c70-4a50-9d90-e01c317856d4 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:54.141009 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4e4f7-0c70-4a50-9d90-e01c317856d4 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:54.141364 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4e4f7-0c70-4a50-9d90-e01c317856d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:54.154548 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682599b-2b8a-42f5-9bb2-ec1569553b48 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:54.156349 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b682599b-2b8a-42f5-9bb2-ec1569553b48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:54.163670 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:54.181494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-6eb4e4f7-0c70-4a50-9d90-e01c317856d4 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:54.183702 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6eb4e4f7-0c70-4a50-9d90-e01c317856d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:54.239473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682599b-2b8a-42f5-9bb2-ec1569553b48 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:54.244856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcd52f-3345-4538-aa25-7e3219161ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:54.245589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcd52f-3345-4538-aa25-7e3219161ec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:54.245880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcd52f-3345-4538-aa25-7e3219161ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:54.246161 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcd52f-3345-4538-aa25-7e3219161ec5 admin admin] PATH_INFO: `/v3/projects/3a82a22030834390b397559317ef19ac` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:54.247390 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcd52f-3345-4538-aa25-7e3219161ec5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a82a22030834390b397559317ef19ac)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:54.248121 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 958/1927] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:54.251454 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcd52f-3345-4538-aa25-7e3219161ec5 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:54.259084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4e4f7-0c70-4a50-9d90-e01c317856d4 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:54.261534 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6eb4e4f7-0c70-4a50-9d90-e01c317856d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:54.284377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:54.301817 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-2bfcd52f-3345-4538-aa25-7e3219161ec5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a82a22030834390b397559317ef19ac', 'request_id': 'req-2bfcd52f-3345-4538-aa25-7e3219161ec5'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:54.309661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-2bfcd52f-3345-4538-aa25-7e3219161ec5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a82a22030834390b397559317ef19ac', 'request_id': 'req-2bfcd52f-3345-4538-aa25-7e3219161ec5'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:54.316571 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-2bfcd52f-3345-4538-aa25-7e3219161ec5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a82a22030834390b397559317ef19ac', 'request_id': 'req-2bfcd52f-3345-4538-aa25-7e3219161ec5'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:54.323518 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4e4f7-0c70-4a50-9d90-e01c317856d4 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:54.329050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-2bfcd52f-3345-4538-aa25-7e3219161ec5 admin admin] The token cache is being invalidate because project 3a82a22030834390b397559317ef19ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:54.329451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-2bfcd52f-3345-4538-aa25-7e3219161ec5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:54.330195 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-2bfcd52f-3345-4538-aa25-7e3219161ec5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:54.331850 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 970/1928] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:54 2022] DELETE /identity/v3/projects/3a82a22030834390b397559317ef19ac => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:36:54.337359 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7927c-d63c-4318-935a-c657eaf32050 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:54.337716 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7927c-d63c-4318-935a-c657eaf32050 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:54.338064 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7927c-d63c-4318-935a-c657eaf32050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:54.359749 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-2aa7927c-d63c-4318-935a-c657eaf32050 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:54.361075 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-2aa7927c-d63c-4318-935a-c657eaf32050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:54.387169 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2b3c3-a42a-4519-a176-a07a877c6251 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:54.388004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2b3c3-a42a-4519-a176-a07a877c6251 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:54.388793 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2b3c3-a42a-4519-a176-a07a877c6251 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:54.389186 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2b3c3-a42a-4519-a176-a07a877c6251 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:54.391015 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-aef2b3c3-a42a-4519-a176-a07a877c6251 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:54.445909 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 971/1929] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:54.454127 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c2f2f-4aa4-4cd9-b1b0-6a8fa86f534f None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:54.454566 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c2f2f-4aa4-4cd9-b1b0-6a8fa86f534f None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:54.454983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c2f2f-4aa4-4cd9-b1b0-6a8fa86f534f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:54.477424 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-004c2f2f-4aa4-4cd9-b1b0-6a8fa86f534f None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:54.478629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-004c2f2f-4aa4-4cd9-b1b0-6a8fa86f534f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:54.478877 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2b3c3-a42a-4519-a176-a07a877c6251 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:54.480326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2b3c3-a42a-4519-a176-a07a877c6251 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:54.489180 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 972/1930] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:54.491372 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 959/1931] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:36:54.493181 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c2f2f-4aa4-4cd9-b1b0-6a8fa86f534f None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:54.502776 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c2f2f-4aa4-4cd9-b1b0-6a8fa86f534f None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:54.507383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03328604-07cf-449a-8faa-da298ed725f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:54.507990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03328604-07cf-449a-8faa-da298ed725f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:54.508267 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03328604-07cf-449a-8faa-da298ed725f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:54.508541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03328604-07cf-449a-8faa-da298ed725f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:54.509260 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03328604-07cf-449a-8faa-da298ed725f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:36:54.509565 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03328604-07cf-449a-8faa-da298ed725f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:54.511166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03328604-07cf-449a-8faa-da298ed725f3 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:54.523688 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 973/1932] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:36:54.527999 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2b3c3-a42a-4519-a176-a07a877c6251 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:54.538269 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2b3c3-a42a-4519-a176-a07a877c6251 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:54.544824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a74bd-97c6-4015-ae6a-8bd096f45e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:54.545699 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a74bd-97c6-4015-ae6a-8bd096f45e5c admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:54.546135 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a74bd-97c6-4015-ae6a-8bd096f45e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:54.546950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a74bd-97c6-4015-ae6a-8bd096f45e5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:54.547226 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a74bd-97c6-4015-ae6a-8bd096f45e5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:54.549171 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a74bd-97c6-4015-ae6a-8bd096f45e5c admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:54.579031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 960/1933] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:54 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:54.583365 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03328604-07cf-449a-8faa-da298ed725f3 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:54.594649 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03328604-07cf-449a-8faa-da298ed725f3 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:54.620627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:54.686133 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe16fe-ce54-4d41-b4c6-492a247f0705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:54.686759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe16fe-ce54-4d41-b4c6-492a247f0705 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:54.687159 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe16fe-ce54-4d41-b4c6-492a247f0705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:54.687447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe16fe-ce54-4d41-b4c6-492a247f0705 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:54.688124 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe16fe-ce54-4d41-b4c6-492a247f0705 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:54.690060 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe16fe-ce54-4d41-b4c6-492a247f0705 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:54.716520 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 974/1934] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:54 2022] POST /identity/v3/users => generated 291 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:54.717803 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a74bd-97c6-4015-ae6a-8bd096f45e5c admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:54.732974 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a74bd-97c6-4015-ae6a-8bd096f45e5c admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:54.737743 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e449e420-e5cd-4146-a9a1-6eba804dfa75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:54.738521 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449e420-e5cd-4146-a9a1-6eba804dfa75 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:54.738637 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449e420-e5cd-4146-a9a1-6eba804dfa75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:54.738906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449e420-e5cd-4146-a9a1-6eba804dfa75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:54.739608 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e449e420-e5cd-4146-a9a1-6eba804dfa75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:54.741754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e449e420-e5cd-4146-a9a1-6eba804dfa75 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:54.755601 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 961/1935] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:54.759483 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe16fe-ce54-4d41-b4c6-492a247f0705 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:54.770888 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe16fe-ce54-4d41-b4c6-492a247f0705 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:54.776586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e797c9a-be18-452a-b86d-bb5caaa38c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:54.777392 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e797c9a-be18-452a-b86d-bb5caaa38c36 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:54.777565 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e797c9a-be18-452a-b86d-bb5caaa38c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:54.777854 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e797c9a-be18-452a-b86d-bb5caaa38c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:54.778731 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e797c9a-be18-452a-b86d-bb5caaa38c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:54.780356 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e797c9a-be18-452a-b86d-bb5caaa38c36 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:54.797038 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 975/1936] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:54.798430 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e449e420-e5cd-4146-a9a1-6eba804dfa75 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:54.808460 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e449e420-e5cd-4146-a9a1-6eba804dfa75 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:54.813746 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78c461-cfd7-46fa-b5d8-0d8aaad13840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:54.814393 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78c461-cfd7-46fa-b5d8-0d8aaad13840 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:54.814674 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78c461-cfd7-46fa-b5d8-0d8aaad13840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:54.814947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78c461-cfd7-46fa-b5d8-0d8aaad13840 admin admin] PATH_INFO: `/v3/projects/043a46fde9154df58774c1ed67a963ab/users/c0b2a741765d4c34b16c4f23a9e0912c/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:54.830088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78c461-cfd7-46fa-b5d8-0d8aaad13840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=043a46fde9154df58774c1ed67a963ab, user_id=c0b2a741765d4c34b16c4f23a9e0912c, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:54.832715 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78c461-cfd7-46fa-b5d8-0d8aaad13840 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:54.847215 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 962/1937] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:54 2022] PUT /identity/v3/projects/043a46fde9154df58774c1ed67a963ab/users/c0b2a741765d4c34b16c4f23a9e0912c/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:54.859170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251d02a-d69b-45bc-82b9-ae3320a7c97a None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:54.859462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251d02a-d69b-45bc-82b9-ae3320a7c97a None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:54.859799 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251d02a-d69b-45bc-82b9-ae3320a7c97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:54.888502 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-6251d02a-d69b-45bc-82b9-ae3320a7c97a None None] MFA Rules not processed for user `c0b2a741765d4c34b16c4f23a9e0912c`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:54.889747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-6251d02a-d69b-45bc-82b9-ae3320a7c97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:54.909448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78c461-cfd7-46fa-b5d8-0d8aaad13840 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:54.911664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-5f78c461-cfd7-46fa-b5d8-0d8aaad13840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:54.948711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:54.972178 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 976/1938] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:54.977052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251d02a-d69b-45bc-82b9-ae3320a7c97a None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:54.995155 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78c461-cfd7-46fa-b5d8-0d8aaad13840 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:55.000073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213343fe-e6f9-446d-9516-fefd15679d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:55.000802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213343fe-e6f9-446d-9516-fefd15679d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:55.001186 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213343fe-e6f9-446d-9516-fefd15679d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:55.001537 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213343fe-e6f9-446d-9516-fefd15679d28 admin admin] PATH_INFO: `/v3/projects/6a344183ca8545908a2a0e6925f8c70d` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:55.002794 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213343fe-e6f9-446d-9516-fefd15679d28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a344183ca8545908a2a0e6925f8c70d)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:55.005164 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213343fe-e6f9-446d-9516-fefd15679d28 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:55.049818 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251d02a-d69b-45bc-82b9-ae3320a7c97a None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:55.058473 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-213343fe-e6f9-446d-9516-fefd15679d28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a344183ca8545908a2a0e6925f8c70d', 'request_id': 'req-213343fe-e6f9-446d-9516-fefd15679d28'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:55.065834 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-213343fe-e6f9-446d-9516-fefd15679d28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a344183ca8545908a2a0e6925f8c70d', 'request_id': 'req-213343fe-e6f9-446d-9516-fefd15679d28'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:55.080542 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-213343fe-e6f9-446d-9516-fefd15679d28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a344183ca8545908a2a0e6925f8c70d', 'request_id': 'req-213343fe-e6f9-446d-9516-fefd15679d28'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:55.083465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:55.095805 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-213343fe-e6f9-446d-9516-fefd15679d28 admin admin] The token cache is being invalidate because project 6a344183ca8545908a2a0e6925f8c70d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:55.096122 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-213343fe-e6f9-446d-9516-fefd15679d28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:55.097683 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-213343fe-e6f9-446d-9516-fefd15679d28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:55.102365 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 963/1939] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:54 2022] DELETE /identity/v3/projects/6a344183ca8545908a2a0e6925f8c70d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:55.106873 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213343fe-e6f9-446d-9516-fefd15679d28 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:55.108802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-213343fe-e6f9-446d-9516-fefd15679d28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:55.124026 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186b7bd-116a-41c4-aad3-a3ff8e733004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:55.124703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186b7bd-116a-41c4-aad3-a3ff8e733004 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:55.125021 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186b7bd-116a-41c4-aad3-a3ff8e733004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:55.125305 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186b7bd-116a-41c4-aad3-a3ff8e733004 admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:55.125981 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186b7bd-116a-41c4-aad3-a3ff8e733004 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:55.127965 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186b7bd-116a-41c4-aad3-a3ff8e733004 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:55.166566 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 977/1940] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:54 2022] POST /identity/v3/projects => generated 381 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:36:55.171181 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186b7bd-116a-41c4-aad3-a3ff8e733004 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:55.173026 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d186b7bd-116a-41c4-aad3-a3ff8e733004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:55.195333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213343fe-e6f9-446d-9516-fefd15679d28 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:55.237600 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186b7bd-116a-41c4-aad3-a3ff8e733004 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:55.242378 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b3faa-2ea3-40d9-bc4f-e186cb187df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:55.242993 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b3faa-2ea3-40d9-bc4f-e186cb187df4 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:55.243274 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b3faa-2ea3-40d9-bc4f-e186cb187df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:55.243552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b3faa-2ea3-40d9-bc4f-e186cb187df4 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:55.244208 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b3faa-2ea3-40d9-bc4f-e186cb187df4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:55.246176 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b3faa-2ea3-40d9-bc4f-e186cb187df4 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:55.254120 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815eda64-142d-413a-a2b4-c859242dc01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:55.254775 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815eda64-142d-413a-a2b4-c859242dc01c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:55.255192 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815eda64-142d-413a-a2b4-c859242dc01c admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:55.255340 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815eda64-142d-413a-a2b4-c859242dc01c admin admin] PATH_INFO: `/v3/users/2f8b140f9bd54211a41e05fd776dcfdf` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:55.256508 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815eda64-142d-413a-a2b4-c859242dc01c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f8b140f9bd54211a41e05fd776dcfdf)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:55.258538 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815eda64-142d-413a-a2b4-c859242dc01c admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:55.263904 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:55.270229 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 978/1941] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:55 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:36:55.274677 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b3faa-2ea3-40d9-bc4f-e186cb187df4 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:55.285804 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b3faa-2ea3-40d9-bc4f-e186cb187df4 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:55.290989 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef7394-a8a5-44bd-a9c1-2971e46fd3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:55.291815 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef7394-a8a5-44bd-a9c1-2971e46fd3d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:55.292218 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef7394-a8a5-44bd-a9c1-2971e46fd3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:55.292541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef7394-a8a5-44bd-a9c1-2971e46fd3d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:55.293425 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef7394-a8a5-44bd-a9c1-2971e46fd3d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:55.295237 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef7394-a8a5-44bd-a9c1-2971e46fd3d6 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:55.306850 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 979/1942] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:55.311916 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef7394-a8a5-44bd-a9c1-2971e46fd3d6 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:55.313371 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-815eda64-142d-413a-a2b4-c859242dc01c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f8b140f9bd54211a41e05fd776dcfdf'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:55.320967 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-815eda64-142d-413a-a2b4-c859242dc01c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f8b140f9bd54211a41e05fd776dcfdf'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:55.321796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef7394-a8a5-44bd-a9c1-2971e46fd3d6 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:55.327000 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70a7dc-69a8-4f86-b1c7-b17864cfef13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:55.327831 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70a7dc-69a8-4f86-b1c7-b17864cfef13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:55.327953 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-815eda64-142d-413a-a2b4-c859242dc01c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f8b140f9bd54211a41e05fd776dcfdf'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:55.328116 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70a7dc-69a8-4f86-b1c7-b17864cfef13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:55.328401 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70a7dc-69a8-4f86-b1c7-b17864cfef13 admin admin] PATH_INFO: `/v3/projects/bce3d9b88f7f4d268cbdac099a959c10/users/102c94fb597c404e9ab50e3dd6dbe548/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:55.340640 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-815eda64-142d-413a-a2b4-c859242dc01c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f8b140f9bd54211a41e05fd776dcfdf'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:55.341830 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-815eda64-142d-413a-a2b4-c859242dc01c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f8b140f9bd54211a41e05fd776dcfdf'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:55.345890 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70a7dc-69a8-4f86-b1c7-b17864cfef13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bce3d9b88f7f4d268cbdac099a959c10, user_id=102c94fb597c404e9ab50e3dd6dbe548, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:55.349581 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70a7dc-69a8-4f86-b1c7-b17864cfef13 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:55.351955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 964/1943] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:55 2022] DELETE /identity/v3/users/2f8b140f9bd54211a41e05fd776dcfdf => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:36:55.366725 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 980/1944] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:55 2022] PUT /identity/v3/projects/bce3d9b88f7f4d268cbdac099a959c10/users/102c94fb597c404e9ab50e3dd6dbe548/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:36:55.371400 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815eda64-142d-413a-a2b4-c859242dc01c admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:55.372990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-815eda64-142d-413a-a2b4-c859242dc01c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:55.442747 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815eda64-142d-413a-a2b4-c859242dc01c admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:55.506592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9c184-cef0-4348-9cf4-c09de39e6462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:55.507365 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9c184-cef0-4348-9cf4-c09de39e6462 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:55.507785 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9c184-cef0-4348-9cf4-c09de39e6462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:55.508114 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9c184-cef0-4348-9cf4-c09de39e6462 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:55.508963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9c184-cef0-4348-9cf4-c09de39e6462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:55.510957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9c184-cef0-4348-9cf4-c09de39e6462 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:55.523546 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 965/1945] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:55.527708 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70a7dc-69a8-4f86-b1c7-b17864cfef13 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:55.539382 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70a7dc-69a8-4f86-b1c7-b17864cfef13 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:55.544257 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cd6ea-d06a-449f-813b-57bc580d92d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:55.544953 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cd6ea-d06a-449f-813b-57bc580d92d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:55.545243 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cd6ea-d06a-449f-813b-57bc580d92d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:55.545522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1cd6ea-d06a-449f-813b-57bc580d92d5 admin admin] PATH_INFO: `/v3/projects/bce3d9b88f7f4d268cbdac099a959c10/users/102c94fb597c404e9ab50e3dd6dbe548/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:55.547925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1cd6ea-d06a-449f-813b-57bc580d92d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bce3d9b88f7f4d268cbdac099a959c10, user_id=102c94fb597c404e9ab50e3dd6dbe548, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:55.550621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1cd6ea-d06a-449f-813b-57bc580d92d5 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:55.561397 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 981/1946] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:55 2022] PUT /identity/v3/projects/bce3d9b88f7f4d268cbdac099a959c10/users/102c94fb597c404e9ab50e3dd6dbe548/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:55.571502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765ff92-269c-426e-8de7-b508a2733db8 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:55.571871 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765ff92-269c-426e-8de7-b508a2733db8 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:55.572215 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765ff92-269c-426e-8de7-b508a2733db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:55.602277 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-6765ff92-269c-426e-8de7-b508a2733db8 None None] MFA Rules not processed for user `102c94fb597c404e9ab50e3dd6dbe548`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:55.603593 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6765ff92-269c-426e-8de7-b508a2733db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:55.628903 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:55.633054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cd6ea-d06a-449f-813b-57bc580d92d5 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:55.636647 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-fb1cd6ea-d06a-449f-813b-57bc580d92d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:55.718329 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 966/1947] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:36:55.732338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765ff92-269c-426e-8de7-b508a2733db8 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:55.734294 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6765ff92-269c-426e-8de7-b508a2733db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:55.743122 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:55.751764 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1cd6ea-d06a-449f-813b-57bc580d92d5 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:55.759358 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f2f3a-969c-41bb-aac7-4e7a6aab56f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:55.760344 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f2f3a-969c-41bb-aac7-4e7a6aab56f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:55.760705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f2f3a-969c-41bb-aac7-4e7a6aab56f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:55.761136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f2f3a-969c-41bb-aac7-4e7a6aab56f3 admin admin] PATH_INFO: `/v3/projects/dd066fe4e9c2459c887a245e80c57694` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:55.763637 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f2f3a-969c-41bb-aac7-4e7a6aab56f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd066fe4e9c2459c887a245e80c57694)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:55.766689 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f2f3a-969c-41bb-aac7-4e7a6aab56f3 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:55.805430 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765ff92-269c-426e-8de7-b508a2733db8 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:55.810795 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2678c1-ee55-4892-b7af-8f3fd6a361d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:55.811499 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2678c1-ee55-4892-b7af-8f3fd6a361d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:55.811875 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2678c1-ee55-4892-b7af-8f3fd6a361d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:55.812202 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2678c1-ee55-4892-b7af-8f3fd6a361d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:55.830943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2678c1-ee55-4892-b7af-8f3fd6a361d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:55.832697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2678c1-ee55-4892-b7af-8f3fd6a361d6 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:55.845385 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1f9f2f3a-969c-41bb-aac7-4e7a6aab56f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd066fe4e9c2459c887a245e80c57694', 'request_id': 'req-1f9f2f3a-969c-41bb-aac7-4e7a6aab56f3'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:55.850161 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 967/1948] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:55 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:36:55.856271 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1f9f2f3a-969c-41bb-aac7-4e7a6aab56f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd066fe4e9c2459c887a245e80c57694', 'request_id': 'req-1f9f2f3a-969c-41bb-aac7-4e7a6aab56f3'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:55.863439 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1f9f2f3a-969c-41bb-aac7-4e7a6aab56f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd066fe4e9c2459c887a245e80c57694', 'request_id': 'req-1f9f2f3a-969c-41bb-aac7-4e7a6aab56f3'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:55.877289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1f9f2f3a-969c-41bb-aac7-4e7a6aab56f3 admin admin] The token cache is being invalidate because project dd066fe4e9c2459c887a245e80c57694 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:55.877852 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1f9f2f3a-969c-41bb-aac7-4e7a6aab56f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:55.878570 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1f9f2f3a-969c-41bb-aac7-4e7a6aab56f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:55.883966 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 982/1949] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:55 2022] DELETE /identity/v3/projects/dd066fe4e9c2459c887a245e80c57694 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:55.886296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49847165-6e73-452a-9e69-2546d47f0017 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:55.886709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49847165-6e73-452a-9e69-2546d47f0017 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:55.887151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49847165-6e73-452a-9e69-2546d47f0017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:55.935618 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-49847165-6e73-452a-9e69-2546d47f0017 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:55.937362 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-49847165-6e73-452a-9e69-2546d47f0017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:56.078192 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 968/1950] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:56.087007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e7953-3081-4119-8eb8-87dd7b93068d None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:56.087516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e7953-3081-4119-8eb8-87dd7b93068d None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:56.087609 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e7953-3081-4119-8eb8-87dd7b93068d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:56.112997 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-075e7953-3081-4119-8eb8-87dd7b93068d None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:56.112997 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-075e7953-3081-4119-8eb8-87dd7b93068d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:56.121845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 983/1951] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:56.126623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49847165-6e73-452a-9e69-2546d47f0017 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:56.137267 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49847165-6e73-452a-9e69-2546d47f0017 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:56.143155 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac490666-846a-4434-b70c-1bfc77fb90df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:56.144078 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac490666-846a-4434-b70c-1bfc77fb90df admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:56.144254 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac490666-846a-4434-b70c-1bfc77fb90df admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:56.144539 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac490666-846a-4434-b70c-1bfc77fb90df admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:56.145286 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac490666-846a-4434-b70c-1bfc77fb90df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:36:56.145596 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac490666-846a-4434-b70c-1bfc77fb90df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:56.147418 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac490666-846a-4434-b70c-1bfc77fb90df admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:56.160722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 969/1952] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:36:56.168500 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e7953-3081-4119-8eb8-87dd7b93068d None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:56.176739 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e7953-3081-4119-8eb8-87dd7b93068d None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:56.186705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c734880-ec44-47be-84ad-111ba343c082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:56.187635 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c734880-ec44-47be-84ad-111ba343c082 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:56.187924 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c734880-ec44-47be-84ad-111ba343c082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:56.188244 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c734880-ec44-47be-84ad-111ba343c082 admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:56.189013 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c734880-ec44-47be-84ad-111ba343c082 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:56.191301 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c734880-ec44-47be-84ad-111ba343c082 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:56.233057 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 984/1953] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:56 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:36:56.235146 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac490666-846a-4434-b70c-1bfc77fb90df admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:56.246124 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac490666-846a-4434-b70c-1bfc77fb90df admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:56.291134 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:56.314604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e164069-fef1-48cd-8c39-7bc344d499bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:56.315160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e164069-fef1-48cd-8c39-7bc344d499bc admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:56.315447 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e164069-fef1-48cd-8c39-7bc344d499bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:56.315908 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e164069-fef1-48cd-8c39-7bc344d499bc admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:56.316575 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e164069-fef1-48cd-8c39-7bc344d499bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:56.319223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e164069-fef1-48cd-8c39-7bc344d499bc admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:56.348971 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 970/1954] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:56 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:56.351269 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c734880-ec44-47be-84ad-111ba343c082 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:56.361652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c734880-ec44-47be-84ad-111ba343c082 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:56.366561 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce5528d-8d09-4022-a644-173216b1d067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:56.367170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5528d-8d09-4022-a644-173216b1d067 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:56.367446 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5528d-8d09-4022-a644-173216b1d067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:56.367740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5528d-8d09-4022-a644-173216b1d067 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:56.368432 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce5528d-8d09-4022-a644-173216b1d067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:56.370037 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce5528d-8d09-4022-a644-173216b1d067 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:56.383434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 985/1955] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:56.384661 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e164069-fef1-48cd-8c39-7bc344d499bc admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:56.392754 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce5528d-8d09-4022-a644-173216b1d067 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:56.394787 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-bce5528d-8d09-4022-a644-173216b1d067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:56.395960 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e164069-fef1-48cd-8c39-7bc344d499bc admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:56.400928 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae52f3-b9fb-423b-9c85-48d25edf4f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:56.401551 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae52f3-b9fb-423b-9c85-48d25edf4f2a admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:56.401836 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae52f3-b9fb-423b-9c85-48d25edf4f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:56.402114 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae52f3-b9fb-423b-9c85-48d25edf4f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:56.402812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae52f3-b9fb-423b-9c85-48d25edf4f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:56.405096 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae52f3-b9fb-423b-9c85-48d25edf4f2a admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:56.408418 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce5528d-8d09-4022-a644-173216b1d067 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:56.413553 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc651b7-5e62-4f23-bde7-aa06da0e5b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:56.414192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc651b7-5e62-4f23-bde7-aa06da0e5b75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:56.414478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc651b7-5e62-4f23-bde7-aa06da0e5b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:56.414781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc651b7-5e62-4f23-bde7-aa06da0e5b75 admin admin] PATH_INFO: `/v3/users/c0b2a741765d4c34b16c4f23a9e0912c` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:56.415955 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc651b7-5e62-4f23-bde7-aa06da0e5b75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0b2a741765d4c34b16c4f23a9e0912c)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:56.419561 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 971/1956] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:56.419703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc651b7-5e62-4f23-bde7-aa06da0e5b75 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:56.420455 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae52f3-b9fb-423b-9c85-48d25edf4f2a admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:56.430233 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae52f3-b9fb-423b-9c85-48d25edf4f2a admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:56.434866 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31250750-b6c5-463d-9119-17dc0e987559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:56.435522 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31250750-b6c5-463d-9119-17dc0e987559 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:56.435804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31250750-b6c5-463d-9119-17dc0e987559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:56.436079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31250750-b6c5-463d-9119-17dc0e987559 admin admin] PATH_INFO: `/v3/projects/c5b3811c9bc04e9e9acc59b68a7a7378/users/00872c87934a4d01a50bfed6564ead3c/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:56.452501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31250750-b6c5-463d-9119-17dc0e987559 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5b3811c9bc04e9e9acc59b68a7a7378, user_id=00872c87934a4d01a50bfed6564ead3c, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:56.455739 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31250750-b6c5-463d-9119-17dc0e987559 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:56.473612 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-bdc651b7-5e62-4f23-bde7-aa06da0e5b75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0b2a741765d4c34b16c4f23a9e0912c'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:56.482265 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 972/1957] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:56 2022] PUT /identity/v3/projects/c5b3811c9bc04e9e9acc59b68a7a7378/users/00872c87934a4d01a50bfed6564ead3c/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:36:56.484551 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-bdc651b7-5e62-4f23-bde7-aa06da0e5b75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0b2a741765d4c34b16c4f23a9e0912c'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:56.494173 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92885b-61db-4417-9a9b-d2fcccfa9fd1 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:56.494780 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92885b-61db-4417-9a9b-d2fcccfa9fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:56.495558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92885b-61db-4417-9a9b-d2fcccfa9fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:56.518674 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-bdc651b7-5e62-4f23-bde7-aa06da0e5b75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0b2a741765d4c34b16c4f23a9e0912c'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:56.535930 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-3c92885b-61db-4417-9a9b-d2fcccfa9fd1 None None] MFA Rules not processed for user `00872c87934a4d01a50bfed6564ead3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:56.537500 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-3c92885b-61db-4417-9a9b-d2fcccfa9fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:56.542412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-bdc651b7-5e62-4f23-bde7-aa06da0e5b75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0b2a741765d4c34b16c4f23a9e0912c'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:56.543322 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-bdc651b7-5e62-4f23-bde7-aa06da0e5b75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0b2a741765d4c34b16c4f23a9e0912c'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:56.551711 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 986/1958] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:56 2022] DELETE /identity/v3/users/c0b2a741765d4c34b16c4f23a9e0912c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:56.556260 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ad499-43ba-4bd9-982d-a3623dd5d859 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:56.556635 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ad499-43ba-4bd9-982d-a3623dd5d859 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:56.557074 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ad499-43ba-4bd9-982d-a3623dd5d859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:56.580981 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-fa0ad499-43ba-4bd9-982d-a3623dd5d859 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:56.582623 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-fa0ad499-43ba-4bd9-982d-a3623dd5d859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:56.596420 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:56.626530 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 973/1959] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:56.631374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92885b-61db-4417-9a9b-d2fcccfa9fd1 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:56.632800 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-3c92885b-61db-4417-9a9b-d2fcccfa9fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:56.657398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 987/1960] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:36:56.668217 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ad499-43ba-4bd9-982d-a3623dd5d859 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:56.669730 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-fa0ad499-43ba-4bd9-982d-a3623dd5d859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:56.714459 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92885b-61db-4417-9a9b-d2fcccfa9fd1 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:56.719534 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff5f01-3dca-4868-b849-7d6de6a84c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:56.720174 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff5f01-3dca-4868-b849-7d6de6a84c66 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:56.720461 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff5f01-3dca-4868-b849-7d6de6a84c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:56.720743 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff5f01-3dca-4868-b849-7d6de6a84c66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:56.721450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff5f01-3dca-4868-b849-7d6de6a84c66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:56.723378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff5f01-3dca-4868-b849-7d6de6a84c66 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:56.738146 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ad499-43ba-4bd9-982d-a3623dd5d859 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:56.742581 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed3092-8ba2-48e8-9e3b-4f3a86809c11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:56.743486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed3092-8ba2-48e8-9e3b-4f3a86809c11 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:56.743762 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed3092-8ba2-48e8-9e3b-4f3a86809c11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:56.744038 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed3092-8ba2-48e8-9e3b-4f3a86809c11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:56.754225 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 974/1961] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:56 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:56.758101 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff5f01-3dca-4868-b849-7d6de6a84c66 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:56.759294 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed3092-8ba2-48e8-9e3b-4f3a86809c11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:56.760754 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed3092-8ba2-48e8-9e3b-4f3a86809c11 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:56.772266 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:56.775545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 988/1962] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:36:56.829564 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff5f01-3dca-4868-b849-7d6de6a84c66 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:56.834864 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42102b-85ef-46a3-95e6-099c57d01de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:56.835489 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42102b-85ef-46a3-95e6-099c57d01de1 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:56.835776 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42102b-85ef-46a3-95e6-099c57d01de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:56.836051 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42102b-85ef-46a3-95e6-099c57d01de1 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:56.836721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c42102b-85ef-46a3-95e6-099c57d01de1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:56.839215 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c42102b-85ef-46a3-95e6-099c57d01de1 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:56.862680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 975/1963] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:56 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:36:56.866161 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed3092-8ba2-48e8-9e3b-4f3a86809c11 service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:56.877008 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed3092-8ba2-48e8-9e3b-4f3a86809c11 service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:56.882490 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da409b-c0aa-46c0-829e-0f438606d4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:56.883264 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da409b-c0aa-46c0-829e-0f438606d4af admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:56.883598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da409b-c0aa-46c0-829e-0f438606d4af admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:56.883884 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da409b-c0aa-46c0-829e-0f438606d4af admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:56.884578 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da409b-c0aa-46c0-829e-0f438606d4af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:56.886284 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da409b-c0aa-46c0-829e-0f438606d4af admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:56.897237 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 989/1964] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:56.901331 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42102b-85ef-46a3-95e6-099c57d01de1 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:56.911917 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42102b-85ef-46a3-95e6-099c57d01de1 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:56.916618 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4479e3-47e8-498f-8f53-c36d3f046b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:56.917252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4479e3-47e8-498f-8f53-c36d3f046b6a admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:56.917536 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4479e3-47e8-498f-8f53-c36d3f046b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:56.917816 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4479e3-47e8-498f-8f53-c36d3f046b6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:56.918506 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4479e3-47e8-498f-8f53-c36d3f046b6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:56.920193 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4479e3-47e8-498f-8f53-c36d3f046b6a admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:56.938106 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 976/1965] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:36:56.942558 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da409b-c0aa-46c0-829e-0f438606d4af admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:56.954955 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da409b-c0aa-46c0-829e-0f438606d4af admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:56.960764 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036816f-d381-495e-9e99-94d4ce112e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:56.961637 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036816f-d381-495e-9e99-94d4ce112e12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:56.962789 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036816f-d381-495e-9e99-94d4ce112e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:56.963176 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036816f-d381-495e-9e99-94d4ce112e12 admin admin] PATH_INFO: `/v3/projects/60a57917f60d43d59d89935fa97d9039/users/b8a43ded9a1b470586612cbd33cd9e75/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:56.982189 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036816f-d381-495e-9e99-94d4ce112e12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60a57917f60d43d59d89935fa97d9039, user_id=b8a43ded9a1b470586612cbd33cd9e75, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:56.984869 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036816f-d381-495e-9e99-94d4ce112e12 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:57.004140 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 990/1966] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:56 2022] PUT /identity/v3/projects/60a57917f60d43d59d89935fa97d9039/users/b8a43ded9a1b470586612cbd33cd9e75/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:36:57.014944 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f140e-4465-46e7-96e1-078edeb5b533 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:57.015280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f140e-4465-46e7-96e1-078edeb5b533 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:57.015559 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f140e-4465-46e7-96e1-078edeb5b533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:57.046318 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-b76f140e-4465-46e7-96e1-078edeb5b533 None None] MFA Rules not processed for user `b8a43ded9a1b470586612cbd33cd9e75`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:57.047460 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b76f140e-4465-46e7-96e1-078edeb5b533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:57.131951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 977/1967] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:57.136247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036816f-d381-495e-9e99-94d4ce112e12 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:57.185387 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f140e-4465-46e7-96e1-078edeb5b533 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:57.186881 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b76f140e-4465-46e7-96e1-078edeb5b533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:57.219939 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036816f-d381-495e-9e99-94d4ce112e12 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:57.227671 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e95d7-3280-471e-a950-0d07cbb3367e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:57.228309 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e95d7-3280-471e-a950-0d07cbb3367e admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:57.228621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e95d7-3280-471e-a950-0d07cbb3367e admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:57.228990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e95d7-3280-471e-a950-0d07cbb3367e admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:57.230373 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e95d7-3280-471e-a950-0d07cbb3367e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:57.231869 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e95d7-3280-471e-a950-0d07cbb3367e admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:57.281491 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 991/1968] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:57 2022] POST /identity/v3/projects => generated 379 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:57.285416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e95d7-3280-471e-a950-0d07cbb3367e admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:57.304406 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f140e-4465-46e7-96e1-078edeb5b533 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:57.304804 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:57.350576 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:57.396183 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e95d7-3280-471e-a950-0d07cbb3367e admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:57.406222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a3fa6-5f97-4488-a5c5-43ec39e2c8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:57.406822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a3fa6-5f97-4488-a5c5-43ec39e2c8d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:57.407105 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a3fa6-5f97-4488-a5c5-43ec39e2c8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:57.407521 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a3fa6-5f97-4488-a5c5-43ec39e2c8d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:57.408318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a3fa6-5f97-4488-a5c5-43ec39e2c8d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:57.410307 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a3fa6-5f97-4488-a5c5-43ec39e2c8d4 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:57.426309 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f8f72-e12e-49c8-99d0-d1d9b9acab59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:57.427080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f8f72-e12e-49c8-99d0-d1d9b9acab59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:57.427489 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f8f72-e12e-49c8-99d0-d1d9b9acab59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:57.427872 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f8f72-e12e-49c8-99d0-d1d9b9acab59 admin admin] PATH_INFO: `/v3/projects/043a46fde9154df58774c1ed67a963ab` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:57.429552 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f8f72-e12e-49c8-99d0-d1d9b9acab59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=043a46fde9154df58774c1ed67a963ab)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:57.433072 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f8f72-e12e-49c8-99d0-d1d9b9acab59 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:57.467924 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 992/1969] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:57 2022] POST /identity/v3/users => generated 296 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:57.471995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a3fa6-5f97-4488-a5c5-43ec39e2c8d4 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:57.484598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a3fa6-5f97-4488-a5c5-43ec39e2c8d4 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:57.491620 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c4ba6-89d8-4a56-abf2-5d576b4c5ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:57.492549 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c4ba6-89d8-4a56-abf2-5d576b4c5ebd admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:57.493007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c4ba6-89d8-4a56-abf2-5d576b4c5ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:57.493497 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c4ba6-89d8-4a56-abf2-5d576b4c5ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:57.494353 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c4ba6-89d8-4a56-abf2-5d576b4c5ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:57.508545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c4ba6-89d8-4a56-abf2-5d576b4c5ebd admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:57.515763 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 993/1970] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:57.519922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-476f8f72-e12e-49c8-99d0-d1d9b9acab59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '043a46fde9154df58774c1ed67a963ab', 'request_id': 'req-476f8f72-e12e-49c8-99d0-d1d9b9acab59'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:57.520157 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c4ba6-89d8-4a56-abf2-5d576b4c5ebd admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:57.525685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-476f8f72-e12e-49c8-99d0-d1d9b9acab59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '043a46fde9154df58774c1ed67a963ab', 'request_id': 'req-476f8f72-e12e-49c8-99d0-d1d9b9acab59'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:57.535872 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c4ba6-89d8-4a56-abf2-5d576b4c5ebd admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:57.537202 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-476f8f72-e12e-49c8-99d0-d1d9b9acab59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '043a46fde9154df58774c1ed67a963ab', 'request_id': 'req-476f8f72-e12e-49c8-99d0-d1d9b9acab59'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:57.550417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-476f8f72-e12e-49c8-99d0-d1d9b9acab59 admin admin] The token cache is being invalidate because project 043a46fde9154df58774c1ed67a963ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:57.550704 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-476f8f72-e12e-49c8-99d0-d1d9b9acab59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:57.551368 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-476f8f72-e12e-49c8-99d0-d1d9b9acab59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:57.553265 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 978/1971] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:57 2022] DELETE /identity/v3/projects/043a46fde9154df58774c1ed67a963ab => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:36:57.556755 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f8f72-e12e-49c8-99d0-d1d9b9acab59 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:57.558131 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-476f8f72-e12e-49c8-99d0-d1d9b9acab59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:57.603132 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec5ce0-27e8-4d29-b048-42d2ef3cb5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:57.603985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec5ce0-27e8-4d29-b048-42d2ef3cb5e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:57.604346 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec5ce0-27e8-4d29-b048-42d2ef3cb5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:57.604829 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec5ce0-27e8-4d29-b048-42d2ef3cb5e0 admin admin] PATH_INFO: `/v3/projects/6093695e389a4239a3bc9ce94d1072d9/users/1e51f3f538d6413c8f96a6c8b28ccc06/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:57.624506 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec5ce0-27e8-4d29-b048-42d2ef3cb5e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6093695e389a4239a3bc9ce94d1072d9, user_id=1e51f3f538d6413c8f96a6c8b28ccc06, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:57.626992 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f8f72-e12e-49c8-99d0-d1d9b9acab59 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:57.629087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec5ce0-27e8-4d29-b048-42d2ef3cb5e0 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:57.632006 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561bb50-9e4a-4f8f-95bd-d4fbd317cfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:57.632727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561bb50-9e4a-4f8f-95bd-d4fbd317cfb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:57.633125 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561bb50-9e4a-4f8f-95bd-d4fbd317cfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:57.633470 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561bb50-9e4a-4f8f-95bd-d4fbd317cfb8 admin admin] PATH_INFO: `/v3/users/102c94fb597c404e9ab50e3dd6dbe548` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:57.634660 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b561bb50-9e4a-4f8f-95bd-d4fbd317cfb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=102c94fb597c404e9ab50e3dd6dbe548)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:57.636946 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b561bb50-9e4a-4f8f-95bd-d4fbd317cfb8 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:57.647971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 994/1972] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:57 2022] PUT /identity/v3/projects/6093695e389a4239a3bc9ce94d1072d9/users/1e51f3f538d6413c8f96a6c8b28ccc06/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:57.651839 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec5ce0-27e8-4d29-b048-42d2ef3cb5e0 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:57.653689 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-92ec5ce0-27e8-4d29-b048-42d2ef3cb5e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:57.695270 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b561bb50-9e4a-4f8f-95bd-d4fbd317cfb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '102c94fb597c404e9ab50e3dd6dbe548'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:57.705249 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b561bb50-9e4a-4f8f-95bd-d4fbd317cfb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '102c94fb597c404e9ab50e3dd6dbe548'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:57.712144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:57.730159 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b561bb50-9e4a-4f8f-95bd-d4fbd317cfb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '102c94fb597c404e9ab50e3dd6dbe548'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:57.764623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b561bb50-9e4a-4f8f-95bd-d4fbd317cfb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '102c94fb597c404e9ab50e3dd6dbe548'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:57.765862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b561bb50-9e4a-4f8f-95bd-d4fbd317cfb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '102c94fb597c404e9ab50e3dd6dbe548'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:57.774073 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec5ce0-27e8-4d29-b048-42d2ef3cb5e0 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:57.783535 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f477e-5819-4040-a8a1-d63279b74ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:57.784332 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f477e-5819-4040-a8a1-d63279b74ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:57.784712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f477e-5819-4040-a8a1-d63279b74ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:57.785116 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f477e-5819-4040-a8a1-d63279b74ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:57.785974 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f477e-5819-4040-a8a1-d63279b74ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:57.787583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 979/1973] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:36:57 2022] DELETE /identity/v3/users/102c94fb597c404e9ab50e3dd6dbe548 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:36:57.787833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f477e-5819-4040-a8a1-d63279b74ea7 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:57.800256 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 995/1974] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:36:57.803992 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561bb50-9e4a-4f8f-95bd-d4fbd317cfb8 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:57.806579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b561bb50-9e4a-4f8f-95bd-d4fbd317cfb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:57.887568 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561bb50-9e4a-4f8f-95bd-d4fbd317cfb8 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:57.893547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b059924-980f-4faf-9949-de09b9f57ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:57.894270 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b059924-980f-4faf-9949-de09b9f57ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:57.894609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b059924-980f-4faf-9949-de09b9f57ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:57.894963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b059924-980f-4faf-9949-de09b9f57ccb admin admin] PATH_INFO: `/v3/projects/6093695e389a4239a3bc9ce94d1072d9/users/1e51f3f538d6413c8f96a6c8b28ccc06/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:57.897477 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b059924-980f-4faf-9949-de09b9f57ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6093695e389a4239a3bc9ce94d1072d9, user_id=1e51f3f538d6413c8f96a6c8b28ccc06, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:57.900270 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b059924-980f-4faf-9949-de09b9f57ccb admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:57.913459 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 980/1975] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:57 2022] PUT /identity/v3/projects/6093695e389a4239a3bc9ce94d1072d9/users/1e51f3f538d6413c8f96a6c8b28ccc06/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:57.926252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a273ab-4202-4993-a82c-24b494b846e0 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:57.926535 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a273ab-4202-4993-a82c-24b494b846e0 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:57.926806 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a273ab-4202-4993-a82c-24b494b846e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:57.950682 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:57.972627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-02a273ab-4202-4993-a82c-24b494b846e0 None None] MFA Rules not processed for user `1e51f3f538d6413c8f96a6c8b28ccc06`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:57.974479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-02a273ab-4202-4993-a82c-24b494b846e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:58.076601 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 996/1976] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:58.094762 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b059924-980f-4faf-9949-de09b9f57ccb admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:58.096822 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0b059924-980f-4faf-9949-de09b9f57ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:58.178890 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b059924-980f-4faf-9949-de09b9f57ccb admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:58.183367 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8959cd-5992-404b-aeb6-50101c5af2e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:58.184068 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8959cd-5992-404b-aeb6-50101c5af2e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:58.184279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8959cd-5992-404b-aeb6-50101c5af2e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:58.184560 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8959cd-5992-404b-aeb6-50101c5af2e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:58.196109 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8959cd-5992-404b-aeb6-50101c5af2e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:58.197606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8959cd-5992-404b-aeb6-50101c5af2e5 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:58.212080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 981/1977] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:58 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:58.256204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a273ab-4202-4993-a82c-24b494b846e0 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:58.258050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-02a273ab-4202-4993-a82c-24b494b846e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:58.272142 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a273ab-4202-4993-a82c-24b494b846e0 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:58.275818 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa5363-6de8-49b9-b37e-16f0d4b29943 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:58.276499 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa5363-6de8-49b9-b37e-16f0d4b29943 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:58.276784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa5363-6de8-49b9-b37e-16f0d4b29943 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:58.277086 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa5363-6de8-49b9-b37e-16f0d4b29943 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:58.279633 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-04aa5363-6de8-49b9-b37e-16f0d4b29943 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:58.338900 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8959cd-5992-404b-aeb6-50101c5af2e5 service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:58.340374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-fd8959cd-5992-404b-aeb6-50101c5af2e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:58.389513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aa5363-6de8-49b9-b37e-16f0d4b29943 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:58.391106 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aa5363-6de8-49b9-b37e-16f0d4b29943 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:58.407401 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 997/1978] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:36:58.484188 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8959cd-5992-404b-aeb6-50101c5af2e5 service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:58.490480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86527ff0-3e19-4324-ae67-70606011dbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:58.491698 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86527ff0-3e19-4324-ae67-70606011dbb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:58.491990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86527ff0-3e19-4324-ae67-70606011dbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:58.492269 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86527ff0-3e19-4324-ae67-70606011dbb0 admin admin] PATH_INFO: `/v3/projects/bce3d9b88f7f4d268cbdac099a959c10` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:58.493791 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86527ff0-3e19-4324-ae67-70606011dbb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bce3d9b88f7f4d268cbdac099a959c10)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:58.495747 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86527ff0-3e19-4324-ae67-70606011dbb0 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:58.520144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:58.564127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-86527ff0-3e19-4324-ae67-70606011dbb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bce3d9b88f7f4d268cbdac099a959c10', 'request_id': 'req-86527ff0-3e19-4324-ae67-70606011dbb0'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:58.570846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-86527ff0-3e19-4324-ae67-70606011dbb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bce3d9b88f7f4d268cbdac099a959c10', 'request_id': 'req-86527ff0-3e19-4324-ae67-70606011dbb0'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:58.589672 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-86527ff0-3e19-4324-ae67-70606011dbb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bce3d9b88f7f4d268cbdac099a959c10', 'request_id': 'req-86527ff0-3e19-4324-ae67-70606011dbb0'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:58.603050 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-86527ff0-3e19-4324-ae67-70606011dbb0 admin admin] The token cache is being invalidate because project bce3d9b88f7f4d268cbdac099a959c10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:36:58.603340 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-86527ff0-3e19-4324-ae67-70606011dbb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:58.604004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-86527ff0-3e19-4324-ae67-70606011dbb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:36:58.610413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 982/1979] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:36:58 2022] DELETE /identity/v3/projects/bce3d9b88f7f4d268cbdac099a959c10 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:36:58.613212 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d68a9-3ab8-44ad-9b49-a1d99fc097e0 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:58.613212 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d68a9-3ab8-44ad-9b49-a1d99fc097e0 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:58.613212 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d68a9-3ab8-44ad-9b49-a1d99fc097e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:58.633403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-bb5d68a9-3ab8-44ad-9b49-a1d99fc097e0 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:58.634716 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-bb5d68a9-3ab8-44ad-9b49-a1d99fc097e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:58.724610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 998/1980] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:58.735161 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa1412-6b73-437e-80fe-6f4263493b1e None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:58.736196 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa1412-6b73-437e-80fe-6f4263493b1e None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:58.737112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa1412-6b73-437e-80fe-6f4263493b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:58.762628 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-ebaa1412-6b73-437e-80fe-6f4263493b1e None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:58.764043 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ebaa1412-6b73-437e-80fe-6f4263493b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:58.803054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 983/1981] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:58.807050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d68a9-3ab8-44ad-9b49-a1d99fc097e0 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:58.825802 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d68a9-3ab8-44ad-9b49-a1d99fc097e0 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:58.831308 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3026d1-7156-4058-8677-71c304be5f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:58.832345 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3026d1-7156-4058-8677-71c304be5f5c admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:58.832780 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3026d1-7156-4058-8677-71c304be5f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:58.833187 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3026d1-7156-4058-8677-71c304be5f5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:58.834247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3026d1-7156-4058-8677-71c304be5f5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:36:58.834660 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3026d1-7156-4058-8677-71c304be5f5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:58.837194 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3026d1-7156-4058-8677-71c304be5f5c admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:58.859545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 999/1982] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:36:58.865715 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa1412-6b73-437e-80fe-6f4263493b1e None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:58.876548 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa1412-6b73-437e-80fe-6f4263493b1e None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:58.883256 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca85bec-817f-4fef-901b-10cfe370c6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:58.883983 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca85bec-817f-4fef-901b-10cfe370c6ea admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:58.884334 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca85bec-817f-4fef-901b-10cfe370c6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:58.884680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca85bec-817f-4fef-901b-10cfe370c6ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:58.885573 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca85bec-817f-4fef-901b-10cfe370c6ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:58.887449 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca85bec-817f-4fef-901b-10cfe370c6ea admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:58.918867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 984/1983] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:58 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:58.923747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3026d1-7156-4058-8677-71c304be5f5c admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:58.934094 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3026d1-7156-4058-8677-71c304be5f5c admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:58.951881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:58.997801 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab167a-7347-4cc8-b011-8d257165f588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:58.998441 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab167a-7347-4cc8-b011-8d257165f588 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:58.998752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab167a-7347-4cc8-b011-8d257165f588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:58.999055 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab167a-7347-4cc8-b011-8d257165f588 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:58.999727 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab167a-7347-4cc8-b011-8d257165f588 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:59.001726 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab167a-7347-4cc8-b011-8d257165f588 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:59.027681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1000/1984] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:58 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:36:59.029603 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca85bec-817f-4fef-901b-10cfe370c6ea admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:59.041227 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca85bec-817f-4fef-901b-10cfe370c6ea admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:59.046682 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e1f25-acb4-4729-a0fe-f84aaf9f1a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:59.047321 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e1f25-acb4-4729-a0fe-f84aaf9f1a42 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:59.047609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e1f25-acb4-4729-a0fe-f84aaf9f1a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:59.047884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e1f25-acb4-4729-a0fe-f84aaf9f1a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:59.048594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e1f25-acb4-4729-a0fe-f84aaf9f1a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:59.050255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e1f25-acb4-4729-a0fe-f84aaf9f1a42 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:59.064278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 985/1985] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:59.065608 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab167a-7347-4cc8-b011-8d257165f588 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:59.076271 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab167a-7347-4cc8-b011-8d257165f588 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:59.081211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a956bdc-1a9b-4bc6-915e-c2c47672999c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:59.081855 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a956bdc-1a9b-4bc6-915e-c2c47672999c admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:59.082141 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a956bdc-1a9b-4bc6-915e-c2c47672999c admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:59.082419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a956bdc-1a9b-4bc6-915e-c2c47672999c admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:59.083123 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a956bdc-1a9b-4bc6-915e-c2c47672999c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:59.084764 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a956bdc-1a9b-4bc6-915e-c2c47672999c admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:59.097541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1001/1986] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:59.102035 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e1f25-acb4-4729-a0fe-f84aaf9f1a42 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:59.113155 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e1f25-acb4-4729-a0fe-f84aaf9f1a42 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:59.118303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f88e9-f21d-4e47-9df1-3db741ab4cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:59.119008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f88e9-f21d-4e47-9df1-3db741ab4cb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:59.119298 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f88e9-f21d-4e47-9df1-3db741ab4cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:59.119578 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f88e9-f21d-4e47-9df1-3db741ab4cb1 admin admin] PATH_INFO: `/v3/projects/5e5666c9be414c85a865c96036b64562/users/fde4b1f47fd34ac783d2874fd43e90c7/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:59.138439 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f88e9-f21d-4e47-9df1-3db741ab4cb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e5666c9be414c85a865c96036b64562, user_id=fde4b1f47fd34ac783d2874fd43e90c7, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:59.142582 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f88e9-f21d-4e47-9df1-3db741ab4cb1 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:59.167481 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 986/1987] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:59 2022] PUT /identity/v3/projects/5e5666c9be414c85a865c96036b64562/users/fde4b1f47fd34ac783d2874fd43e90c7/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:36:59.178473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1c064-9507-4a45-9874-a21305274aba None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:59.178787 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1c064-9507-4a45-9874-a21305274aba None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:59.179736 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1c064-9507-4a45-9874-a21305274aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:59.209747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-0ef1c064-9507-4a45-9874-a21305274aba None None] MFA Rules not processed for user `fde4b1f47fd34ac783d2874fd43e90c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:59.210957 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-0ef1c064-9507-4a45-9874-a21305274aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:59.298168 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1002/1988] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:36:59.303042 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f88e9-f21d-4e47-9df1-3db741ab4cb1 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:59.376561 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f88e9-f21d-4e47-9df1-3db741ab4cb1 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:59.381295 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77541e11-77f6-44db-8663-693827389848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:59.381925 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77541e11-77f6-44db-8663-693827389848 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:59.382211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77541e11-77f6-44db-8663-693827389848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:59.382492 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77541e11-77f6-44db-8663-693827389848 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:59.383161 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77541e11-77f6-44db-8663-693827389848 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:59.385135 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77541e11-77f6-44db-8663-693827389848 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:59.417489 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 987/1989] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:36:59 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:59.418698 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1c064-9507-4a45-9874-a21305274aba None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:59.430655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1c064-9507-4a45-9874-a21305274aba None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:59.516661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70aa266-2af1-4967-a8b6-270bc80c6de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:59.517348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70aa266-2af1-4967-a8b6-270bc80c6de7 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:59.517636 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70aa266-2af1-4967-a8b6-270bc80c6de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:59.517916 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70aa266-2af1-4967-a8b6-270bc80c6de7 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:59.518610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70aa266-2af1-4967-a8b6-270bc80c6de7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:59.520538 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70aa266-2af1-4967-a8b6-270bc80c6de7 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:59.545795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1003/1990] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:36:59 2022] POST /identity/v3/users => generated 299 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:36:59.547011 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77541e11-77f6-44db-8663-693827389848 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:59.557519 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77541e11-77f6-44db-8663-693827389848 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:59.562743 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1a7d1-a552-4ffe-ba30-de19423d0280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:59.563458 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1a7d1-a552-4ffe-ba30-de19423d0280 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:59.563642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1a7d1-a552-4ffe-ba30-de19423d0280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:59.563919 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1a7d1-a552-4ffe-ba30-de19423d0280 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:59.564655 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1a7d1-a552-4ffe-ba30-de19423d0280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:59.566301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1a7d1-a552-4ffe-ba30-de19423d0280 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:59.578392 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 988/1991] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:36:59.582847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70aa266-2af1-4967-a8b6-270bc80c6de7 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:59.593686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70aa266-2af1-4967-a8b6-270bc80c6de7 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:59.598545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2be3b2-bde5-4b04-b76c-7e547f8c213d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:59.599528 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2be3b2-bde5-4b04-b76c-7e547f8c213d admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:59.599645 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2be3b2-bde5-4b04-b76c-7e547f8c213d admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:59.600068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2be3b2-bde5-4b04-b76c-7e547f8c213d admin admin] PATH_INFO: `/v3/projects/c74ca0fbb5a1491db93cd252a975e6b6/users/08d670fd660c43c4b55addc5b1f636a1/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:59.618221 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2be3b2-bde5-4b04-b76c-7e547f8c213d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c74ca0fbb5a1491db93cd252a975e6b6, user_id=08d670fd660c43c4b55addc5b1f636a1, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:59.621023 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2be3b2-bde5-4b04-b76c-7e547f8c213d admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:59.623939 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:59.644940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1004/1992] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:59 2022] PUT /identity/v3/projects/c74ca0fbb5a1491db93cd252a975e6b6/users/08d670fd660c43c4b55addc5b1f636a1/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:59.646144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1a7d1-a552-4ffe-ba30-de19423d0280 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:59.657889 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1a7d1-a552-4ffe-ba30-de19423d0280 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:59.669471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:36:59.736614 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609905e-656a-4ceb-853f-1c5ab7015acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:59.737776 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609905e-656a-4ceb-853f-1c5ab7015acd admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:59.738132 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609905e-656a-4ceb-853f-1c5ab7015acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:59.739382 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609905e-656a-4ceb-853f-1c5ab7015acd admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:59.740180 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609905e-656a-4ceb-853f-1c5ab7015acd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:59.741982 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609905e-656a-4ceb-853f-1c5ab7015acd admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:59.755369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 989/1993] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:36:59.759661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2be3b2-bde5-4b04-b76c-7e547f8c213d admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:59.771954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2be3b2-bde5-4b04-b76c-7e547f8c213d admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:36:59.777169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2c1b6-5bf0-4b2c-ab65-51251d7c7553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:36:59.778031 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2c1b6-5bf0-4b2c-ab65-51251d7c7553 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:59.779212 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2c1b6-5bf0-4b2c-ab65-51251d7c7553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:59.780314 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2c1b6-5bf0-4b2c-ab65-51251d7c7553 admin admin] PATH_INFO: `/v3/projects/c74ca0fbb5a1491db93cd252a975e6b6/users/08d670fd660c43c4b55addc5b1f636a1/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:59.791022 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb2c1b6-5bf0-4b2c-ab65-51251d7c7553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c74ca0fbb5a1491db93cd252a975e6b6, user_id=08d670fd660c43c4b55addc5b1f636a1, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:36:59.794109 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb2c1b6-5bf0-4b2c-ab65-51251d7c7553 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:36:59.807912 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1005/1994] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:36:59 2022] PUT /identity/v3/projects/c74ca0fbb5a1491db93cd252a975e6b6/users/08d670fd660c43c4b55addc5b1f636a1/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:36:59.818423 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e62b1-20e0-4691-a23c-a37abd04323b None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:36:59.818765 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e62b1-20e0-4691-a23c-a37abd04323b None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:36:59.819085 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e62b1-20e0-4691-a23c-a37abd04323b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:36:59.851561 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-b08e62b1-20e0-4691-a23c-a37abd04323b None None] MFA Rules not processed for user `08d670fd660c43c4b55addc5b1f636a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:36:59.852440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b08e62b1-20e0-4691-a23c-a37abd04323b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:36:59.969337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 990/1995] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:36:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:36:59.991573 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2c1b6-5bf0-4b2c-ab65-51251d7c7553 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:36:59.993356 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-adb2c1b6-5bf0-4b2c-ab65-51251d7c7553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:00.051990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2c1b6-5bf0-4b2c-ab65-51251d7c7553 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:00.055880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911e5d4-0d43-4fd1-9eb9-24a6c2ba4476 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:00.056758 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911e5d4-0d43-4fd1-9eb9-24a6c2ba4476 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:00.056877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911e5d4-0d43-4fd1-9eb9-24a6c2ba4476 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:00.057245 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911e5d4-0d43-4fd1-9eb9-24a6c2ba4476 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:00.130110 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911e5d4-0d43-4fd1-9eb9-24a6c2ba4476 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:00.132246 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911e5d4-0d43-4fd1-9eb9-24a6c2ba4476 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:00.144124 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1006/1996] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:36:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:37:02.103581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e62b1-20e0-4691-a23c-a37abd04323b None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:02.105153 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b08e62b1-20e0-4691-a23c-a37abd04323b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:02.201431 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e62b1-20e0-4691-a23c-a37abd04323b None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:02.207210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d762961-2f28-4317-a694-17b15f6f9f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:02.207976 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d762961-2f28-4317-a694-17b15f6f9f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:02.208917 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d762961-2f28-4317-a694-17b15f6f9f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:02.208917 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d762961-2f28-4317-a694-17b15f6f9f6b admin admin] PATH_INFO: `/v3/users/00872c87934a4d01a50bfed6564ead3c` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:02.211171 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d762961-2f28-4317-a694-17b15f6f9f6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=00872c87934a4d01a50bfed6564ead3c)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:02.213894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d762961-2f28-4317-a694-17b15f6f9f6b admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:02.273895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:02.292300 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-5d762961-2f28-4317-a694-17b15f6f9f6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00872c87934a4d01a50bfed6564ead3c'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:02.303435 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-5d762961-2f28-4317-a694-17b15f6f9f6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00872c87934a4d01a50bfed6564ead3c'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:02.328834 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-5d762961-2f28-4317-a694-17b15f6f9f6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00872c87934a4d01a50bfed6564ead3c'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:02.343998 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-5d762961-2f28-4317-a694-17b15f6f9f6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00872c87934a4d01a50bfed6564ead3c'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:02.344972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-5d762961-2f28-4317-a694-17b15f6f9f6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00872c87934a4d01a50bfed6564ead3c'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:02.352390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 991/1997] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:37:02 2022] DELETE /identity/v3/users/00872c87934a4d01a50bfed6564ead3c => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:02.357396 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807d3ec-b029-4efa-8124-1c61e4c47007 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:02.357877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807d3ec-b029-4efa-8124-1c61e4c47007 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:02.357997 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807d3ec-b029-4efa-8124-1c61e4c47007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:02.394451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-e807d3ec-b029-4efa-8124-1c61e4c47007 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:02.396466 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e807d3ec-b029-4efa-8124-1c61e4c47007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:02.473620 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1007/1998] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:02.484092 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d762961-2f28-4317-a694-17b15f6f9f6b admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:02.485561 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-5d762961-2f28-4317-a694-17b15f6f9f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:02.550823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d762961-2f28-4317-a694-17b15f6f9f6b admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:02.555603 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f05d10-5740-4529-9a6e-8c8d5c29328b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:02.556785 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f05d10-5740-4529-9a6e-8c8d5c29328b service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:02.557425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f05d10-5740-4529-9a6e-8c8d5c29328b service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:02.557998 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f05d10-5740-4529-9a6e-8c8d5c29328b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:02.575561 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f05d10-5740-4529-9a6e-8c8d5c29328b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:02.577258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f05d10-5740-4529-9a6e-8c8d5c29328b service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:02.592166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 992/1999] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:37:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:37:02.872227 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807d3ec-b029-4efa-8124-1c61e4c47007 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:02.873855 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e807d3ec-b029-4efa-8124-1c61e4c47007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:02.890172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807d3ec-b029-4efa-8124-1c61e4c47007 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:02.897236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad8c95-9fd9-4e32-84a9-6c626f45436a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:02.898325 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad8c95-9fd9-4e32-84a9-6c626f45436a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:02.898821 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad8c95-9fd9-4e32-84a9-6c626f45436a admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:02.899312 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad8c95-9fd9-4e32-84a9-6c626f45436a admin admin] PATH_INFO: `/v3/projects/c5b3811c9bc04e9e9acc59b68a7a7378` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:02.901366 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad8c95-9fd9-4e32-84a9-6c626f45436a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5b3811c9bc04e9e9acc59b68a7a7378)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:02.905595 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad8c95-9fd9-4e32-84a9-6c626f45436a admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:02.944671 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:INSERT=2,keystone:SELECT=96,keystone:DELETE=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:02.961703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f6ad8c95-9fd9-4e32-84a9-6c626f45436a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5b3811c9bc04e9e9acc59b68a7a7378', 'request_id': 'req-f6ad8c95-9fd9-4e32-84a9-6c626f45436a'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:02.970124 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f6ad8c95-9fd9-4e32-84a9-6c626f45436a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5b3811c9bc04e9e9acc59b68a7a7378', 'request_id': 'req-f6ad8c95-9fd9-4e32-84a9-6c626f45436a'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:02.979180 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f6ad8c95-9fd9-4e32-84a9-6c626f45436a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5b3811c9bc04e9e9acc59b68a7a7378', 'request_id': 'req-f6ad8c95-9fd9-4e32-84a9-6c626f45436a'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:02.993494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f6ad8c95-9fd9-4e32-84a9-6c626f45436a admin admin] The token cache is being invalidate because project c5b3811c9bc04e9e9acc59b68a7a7378 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:37:02.993494 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f6ad8c95-9fd9-4e32-84a9-6c626f45436a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:02.994322 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-f6ad8c95-9fd9-4e32-84a9-6c626f45436a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:02.995947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1008/2000] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:37:02 2022] DELETE /identity/v3/projects/c5b3811c9bc04e9e9acc59b68a7a7378 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:03.000149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f05d10-5740-4529-9a6e-8c8d5c29328b service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:03.012528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f05d10-5740-4529-9a6e-8c8d5c29328b service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:03.073232 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c7798-fd7e-4117-9c39-6c0e75d9f717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:03.073881 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c7798-fd7e-4117-9c39-6c0e75d9f717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:03.074162 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c7798-fd7e-4117-9c39-6c0e75d9f717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:03.074440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c7798-fd7e-4117-9c39-6c0e75d9f717 admin admin] PATH_INFO: `/v3/users/b8a43ded9a1b470586612cbd33cd9e75` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:03.076134 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c7798-fd7e-4117-9c39-6c0e75d9f717 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8a43ded9a1b470586612cbd33cd9e75)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:03.078089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c7798-fd7e-4117-9c39-6c0e75d9f717 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:03.139534 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8b1c7798-fd7e-4117-9c39-6c0e75d9f717 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8a43ded9a1b470586612cbd33cd9e75'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:03.146552 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8b1c7798-fd7e-4117-9c39-6c0e75d9f717 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8a43ded9a1b470586612cbd33cd9e75'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:03.160315 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:03.165517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8b1c7798-fd7e-4117-9c39-6c0e75d9f717 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8a43ded9a1b470586612cbd33cd9e75'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:03.180484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8b1c7798-fd7e-4117-9c39-6c0e75d9f717 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8a43ded9a1b470586612cbd33cd9e75'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:03.181250 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8b1c7798-fd7e-4117-9c39-6c0e75d9f717 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8a43ded9a1b470586612cbd33cd9e75'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:03.187312 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 993/2001] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:37:02 2022] DELETE /identity/v3/users/b8a43ded9a1b470586612cbd33cd9e75 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:03.899990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad8c95-9fd9-4e32-84a9-6c626f45436a admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:03.912939 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f6ad8c95-9fd9-4e32-84a9-6c626f45436a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:03.936740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad8c95-9fd9-4e32-84a9-6c626f45436a admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:04.009233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c973b-90fa-4f1d-96cf-13c03fff7f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:04.009819 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c973b-90fa-4f1d-96cf-13c03fff7f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:04.010100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c973b-90fa-4f1d-96cf-13c03fff7f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:04.010379 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c973b-90fa-4f1d-96cf-13c03fff7f1c admin admin] PATH_INFO: `/v3/projects/60a57917f60d43d59d89935fa97d9039` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:04.011675 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c973b-90fa-4f1d-96cf-13c03fff7f1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=60a57917f60d43d59d89935fa97d9039)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:04.013855 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c973b-90fa-4f1d-96cf-13c03fff7f1c admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:04.070608 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ec4c973b-90fa-4f1d-96cf-13c03fff7f1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60a57917f60d43d59d89935fa97d9039', 'request_id': 'req-ec4c973b-90fa-4f1d-96cf-13c03fff7f1c'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:04.080621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ec4c973b-90fa-4f1d-96cf-13c03fff7f1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60a57917f60d43d59d89935fa97d9039', 'request_id': 'req-ec4c973b-90fa-4f1d-96cf-13c03fff7f1c'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:04.088201 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ec4c973b-90fa-4f1d-96cf-13c03fff7f1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60a57917f60d43d59d89935fa97d9039', 'request_id': 'req-ec4c973b-90fa-4f1d-96cf-13c03fff7f1c'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:04.101982 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ec4c973b-90fa-4f1d-96cf-13c03fff7f1c admin admin] The token cache is being invalidate because project 60a57917f60d43d59d89935fa97d9039 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:37:04.102452 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ec4c973b-90fa-4f1d-96cf-13c03fff7f1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:04.103479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ec4c973b-90fa-4f1d-96cf-13c03fff7f1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:04.105124 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1009/2002] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:37:03 2022] DELETE /identity/v3/projects/60a57917f60d43d59d89935fa97d9039 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:04.109741 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c7798-fd7e-4117-9c39-6c0e75d9f717 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:04.111726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-8b1c7798-fd7e-4117-9c39-6c0e75d9f717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:04.129404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c7798-fd7e-4117-9c39-6c0e75d9f717 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:04.212418 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba60b63-d71c-4c0a-8400-d626b9c2b738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:04.213168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba60b63-d71c-4c0a-8400-d626b9c2b738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:04.213686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba60b63-d71c-4c0a-8400-d626b9c2b738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:04.214130 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba60b63-d71c-4c0a-8400-d626b9c2b738 admin admin] PATH_INFO: `/v3/users/1e51f3f538d6413c8f96a6c8b28ccc06` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:04.215350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba60b63-d71c-4c0a-8400-d626b9c2b738 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e51f3f538d6413c8f96a6c8b28ccc06)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:04.218019 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba60b63-d71c-4c0a-8400-d626b9c2b738 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:04.285373 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-7ba60b63-d71c-4c0a-8400-d626b9c2b738 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e51f3f538d6413c8f96a6c8b28ccc06'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:04.294899 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-7ba60b63-d71c-4c0a-8400-d626b9c2b738 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e51f3f538d6413c8f96a6c8b28ccc06'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:04.303969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-7ba60b63-d71c-4c0a-8400-d626b9c2b738 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e51f3f538d6413c8f96a6c8b28ccc06'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:04.318394 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-7ba60b63-d71c-4c0a-8400-d626b9c2b738 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e51f3f538d6413c8f96a6c8b28ccc06'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:04.319469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-7ba60b63-d71c-4c0a-8400-d626b9c2b738 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e51f3f538d6413c8f96a6c8b28ccc06'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:04.326419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 994/2003] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:37:04 2022] DELETE /identity/v3/users/1e51f3f538d6413c8f96a6c8b28ccc06 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:04.938643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c973b-90fa-4f1d-96cf-13c03fff7f1c admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:04.940930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-ec4c973b-90fa-4f1d-96cf-13c03fff7f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:04.968510 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba60b63-d71c-4c0a-8400-d626b9c2b738 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:04.969895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-7ba60b63-d71c-4c0a-8400-d626b9c2b738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:05.005455 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:05.011083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c973b-90fa-4f1d-96cf-13c03fff7f1c admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:05.071128 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:05.077712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fec42-0d4a-418f-9b4a-e14e8063a74c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:05.078627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fec42-0d4a-418f-9b4a-e14e8063a74c service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:05.079252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fec42-0d4a-418f-9b4a-e14e8063a74c service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:05.079704 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fec42-0d4a-418f-9b4a-e14e8063a74c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:05.080077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba60b63-d71c-4c0a-8400-d626b9c2b738 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:05.083961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-602fec42-0d4a-418f-9b4a-e14e8063a74c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:05.093727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4245b71-8de3-4b18-81dd-477acc9dc319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:05.094915 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4245b71-8de3-4b18-81dd-477acc9dc319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:05.095861 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4245b71-8de3-4b18-81dd-477acc9dc319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:05.096390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4245b71-8de3-4b18-81dd-477acc9dc319 admin admin] PATH_INFO: `/v3/projects/6093695e389a4239a3bc9ce94d1072d9` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:05.098113 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4245b71-8de3-4b18-81dd-477acc9dc319 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6093695e389a4239a3bc9ce94d1072d9)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:05.100492 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4245b71-8de3-4b18-81dd-477acc9dc319 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:05.165721 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fec42-0d4a-418f-9b4a-e14e8063a74c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:05.165721 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fec42-0d4a-418f-9b4a-e14e8063a74c service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:05.170406 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a4245b71-8de3-4b18-81dd-477acc9dc319 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6093695e389a4239a3bc9ce94d1072d9', 'request_id': 'req-a4245b71-8de3-4b18-81dd-477acc9dc319'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:05.177290 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a4245b71-8de3-4b18-81dd-477acc9dc319 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6093695e389a4239a3bc9ce94d1072d9', 'request_id': 'req-a4245b71-8de3-4b18-81dd-477acc9dc319'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:05.177426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1010/2004] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:37:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:37:05.188204 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a4245b71-8de3-4b18-81dd-477acc9dc319 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6093695e389a4239a3bc9ce94d1072d9', 'request_id': 'req-a4245b71-8de3-4b18-81dd-477acc9dc319'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:05.202525 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a4245b71-8de3-4b18-81dd-477acc9dc319 admin admin] The token cache is being invalidate because project 6093695e389a4239a3bc9ce94d1072d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:37:05.202829 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a4245b71-8de3-4b18-81dd-477acc9dc319 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:05.203490 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a4245b71-8de3-4b18-81dd-477acc9dc319 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:05.205332 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 995/2005] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:37:04 2022] DELETE /identity/v3/projects/6093695e389a4239a3bc9ce94d1072d9 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:05.211144 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8718fb-3043-435a-861d-6ce2ded42ae5 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:05.211440 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8718fb-3043-435a-861d-6ce2ded42ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:05.211723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8718fb-3043-435a-861d-6ce2ded42ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:05.240247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-cd8718fb-3043-435a-861d-6ce2ded42ae5 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:05.241639 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-cd8718fb-3043-435a-861d-6ce2ded42ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:05.347331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1011/2006] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:05.355405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948336b-fdcd-495c-b425-0a5939fbb8ae None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:05.355847 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948336b-fdcd-495c-b425-0a5939fbb8ae None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:05.355999 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948336b-fdcd-495c-b425-0a5939fbb8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:05.381695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-d948336b-fdcd-495c-b425-0a5939fbb8ae None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:05.383073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d948336b-fdcd-495c-b425-0a5939fbb8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:05.393370 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 996/2007] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:05.397895 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8718fb-3043-435a-861d-6ce2ded42ae5 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:05.415928 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8718fb-3043-435a-861d-6ce2ded42ae5 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:05.421041 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039b2da-df5a-4bdc-ab11-266faac8824a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:05.422052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039b2da-df5a-4bdc-ab11-266faac8824a admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:05.422487 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039b2da-df5a-4bdc-ab11-266faac8824a admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:05.422912 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039b2da-df5a-4bdc-ab11-266faac8824a admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:05.423855 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039b2da-df5a-4bdc-ab11-266faac8824a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:37:05.424254 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039b2da-df5a-4bdc-ab11-266faac8824a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:05.426309 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039b2da-df5a-4bdc-ab11-266faac8824a admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:05.440812 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1012/2008] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:37:05.446007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948336b-fdcd-495c-b425-0a5939fbb8ae None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:05.460735 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948336b-fdcd-495c-b425-0a5939fbb8ae None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:05.469895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9a664-a58d-4fd3-8374-e3a14910dfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:05.470615 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9a664-a58d-4fd3-8374-e3a14910dfcd admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:05.470976 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9a664-a58d-4fd3-8374-e3a14910dfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:05.471322 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9a664-a58d-4fd3-8374-e3a14910dfcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:05.472495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c9a664-a58d-4fd3-8374-e3a14910dfcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:05.475456 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c9a664-a58d-4fd3-8374-e3a14910dfcd admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:05.516573 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 997/2009] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:37:05 2022] POST /identity/v3/projects => generated 365 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:37:05.520634 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039b2da-df5a-4bdc-ab11-266faac8824a admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:05.539370 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039b2da-df5a-4bdc-ab11-266faac8824a admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:05.625054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:05.679373 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c471ce-8935-4b19-9b10-9cbf74057713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:05.680259 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c471ce-8935-4b19-9b10-9cbf74057713 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:05.680673 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c471ce-8935-4b19-9b10-9cbf74057713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:05.681094 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c471ce-8935-4b19-9b10-9cbf74057713 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:05.682266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c471ce-8935-4b19-9b10-9cbf74057713 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:05.685425 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c471ce-8935-4b19-9b10-9cbf74057713 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:05.727750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1013/2010] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:37:05 2022] POST /identity/v3/users => generated 283 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:37:05.728422 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9a664-a58d-4fd3-8374-e3a14910dfcd admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:05.741733 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9a664-a58d-4fd3-8374-e3a14910dfcd admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:05.746730 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a49152-6917-400b-98a7-2a44a9e9b522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:05.747884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a49152-6917-400b-98a7-2a44a9e9b522 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:05.748171 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a49152-6917-400b-98a7-2a44a9e9b522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:05.748446 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a49152-6917-400b-98a7-2a44a9e9b522 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:05.749312 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a49152-6917-400b-98a7-2a44a9e9b522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:05.750952 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a49152-6917-400b-98a7-2a44a9e9b522 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:05.763540 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 998/2011] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:37:05.768380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c471ce-8935-4b19-9b10-9cbf74057713 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:05.780244 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c471ce-8935-4b19-9b10-9cbf74057713 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:05.792057 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671c1e4-6de6-47ba-ab25-852cca3bedd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:05.793592 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671c1e4-6de6-47ba-ab25-852cca3bedd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:05.794683 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671c1e4-6de6-47ba-ab25-852cca3bedd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:05.795153 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671c1e4-6de6-47ba-ab25-852cca3bedd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:05.797106 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671c1e4-6de6-47ba-ab25-852cca3bedd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:05.798893 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671c1e4-6de6-47ba-ab25-852cca3bedd8 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:05.814125 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1014/2012] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:37:05.818238 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a49152-6917-400b-98a7-2a44a9e9b522 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:05.830689 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a49152-6917-400b-98a7-2a44a9e9b522 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:05.836202 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe85ce-f94d-4e3b-b027-b38e08348ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:05.836355 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe85ce-f94d-4e3b-b027-b38e08348ac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:05.836602 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe85ce-f94d-4e3b-b027-b38e08348ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:05.836934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe85ce-f94d-4e3b-b027-b38e08348ac1 admin admin] PATH_INFO: `/v3/projects/113def71e1724f5e8a776a63bca122f1/users/487b854cb9ba4fc2ad732061f6772b5b/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:05.858061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe85ce-f94d-4e3b-b027-b38e08348ac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=113def71e1724f5e8a776a63bca122f1, user_id=487b854cb9ba4fc2ad732061f6772b5b, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:05.860880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe85ce-f94d-4e3b-b027-b38e08348ac1 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:05.877839 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 999/2013] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:37:05 2022] PUT /identity/v3/projects/113def71e1724f5e8a776a63bca122f1/users/487b854cb9ba4fc2ad732061f6772b5b/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:37:05.890820 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cbb92-d0aa-474e-9603-c219af1f9a36 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:05.891107 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cbb92-d0aa-474e-9603-c219af1f9a36 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:05.891802 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cbb92-d0aa-474e-9603-c219af1f9a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:05.942827 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-8c0cbb92-d0aa-474e-9603-c219af1f9a36 None None] MFA Rules not processed for user `487b854cb9ba4fc2ad732061f6772b5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:05.945322 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-8c0cbb92-d0aa-474e-9603-c219af1f9a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:06.038578 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1015/2014] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:06.040043 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe85ce-f94d-4e3b-b027-b38e08348ac1 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:06.110713 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe85ce-f94d-4e3b-b027-b38e08348ac1 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:06.116241 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732228f8-9b90-4b19-a5cb-22d3edbd81e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:06.117173 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732228f8-9b90-4b19-a5cb-22d3edbd81e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:06.117526 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732228f8-9b90-4b19-a5cb-22d3edbd81e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:06.118073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732228f8-9b90-4b19-a5cb-22d3edbd81e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:06.118823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732228f8-9b90-4b19-a5cb-22d3edbd81e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:06.121426 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732228f8-9b90-4b19-a5cb-22d3edbd81e4 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:06.165277 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1000/2015] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:37:06 2022] POST /identity/v3/projects => generated 365 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:37:06.171913 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0cbb92-d0aa-474e-9603-c219af1f9a36 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:06.185534 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0cbb92-d0aa-474e-9603-c219af1f9a36 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:06.286069 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca33e4f-accd-495b-8089-6240eead64c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:06.286926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca33e4f-accd-495b-8089-6240eead64c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:06.287191 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca33e4f-accd-495b-8089-6240eead64c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:06.287988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca33e4f-accd-495b-8089-6240eead64c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:06.288784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca33e4f-accd-495b-8089-6240eead64c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:06.290872 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca33e4f-accd-495b-8089-6240eead64c0 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:06.323760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1016/2016] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:37:06 2022] POST /identity/v3/users => generated 289 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:37:06.324059 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732228f8-9b90-4b19-a5cb-22d3edbd81e4 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:06.336676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732228f8-9b90-4b19-a5cb-22d3edbd81e4 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:06.345783 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772bf60-53fa-43b1-a65a-0a655c65ba68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:06.347190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772bf60-53fa-43b1-a65a-0a655c65ba68 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:06.347561 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772bf60-53fa-43b1-a65a-0a655c65ba68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:06.347917 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772bf60-53fa-43b1-a65a-0a655c65ba68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:06.349293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e772bf60-53fa-43b1-a65a-0a655c65ba68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:06.351373 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e772bf60-53fa-43b1-a65a-0a655c65ba68 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:06.364911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:06.376313 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1001/2017] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:37:06.377264 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca33e4f-accd-495b-8089-6240eead64c0 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:06.387461 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca33e4f-accd-495b-8089-6240eead64c0 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:06.395568 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eed78d-e4ca-41c9-a068-bae928a9cd09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:06.396282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eed78d-e4ca-41c9-a068-bae928a9cd09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:06.396590 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eed78d-e4ca-41c9-a068-bae928a9cd09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:06.396873 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eed78d-e4ca-41c9-a068-bae928a9cd09 admin admin] PATH_INFO: `/v3/projects/5d1057cd915f42ed9744e648d7ff579a/users/5a7e8c9d48ad4266b46e9e0353c46579/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:06.424122 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eed78d-e4ca-41c9-a068-bae928a9cd09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d1057cd915f42ed9744e648d7ff579a, user_id=5a7e8c9d48ad4266b46e9e0353c46579, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:06.428502 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eed78d-e4ca-41c9-a068-bae928a9cd09 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:06.447037 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1017/2018] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:37:06 2022] PUT /identity/v3/projects/5d1057cd915f42ed9744e648d7ff579a/users/5a7e8c9d48ad4266b46e9e0353c46579/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:06.451466 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772bf60-53fa-43b1-a65a-0a655c65ba68 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:06.461767 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772bf60-53fa-43b1-a65a-0a655c65ba68 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:06.535731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408ff67-b3d1-4fa1-ba14-fd496637e1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:06.536388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408ff67-b3d1-4fa1-ba14-fd496637e1d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:06.536680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408ff67-b3d1-4fa1-ba14-fd496637e1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:06.536998 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408ff67-b3d1-4fa1-ba14-fd496637e1d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:06.537694 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408ff67-b3d1-4fa1-ba14-fd496637e1d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:06.539305 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408ff67-b3d1-4fa1-ba14-fd496637e1d9 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:06.550824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1002/2019] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:37:06.555441 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eed78d-e4ca-41c9-a068-bae928a9cd09 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:06.566870 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eed78d-e4ca-41c9-a068-bae928a9cd09 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:06.571845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934de6e5-802f-4d6d-8929-ab2f69dcd861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:06.572606 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934de6e5-802f-4d6d-8929-ab2f69dcd861 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:06.572989 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934de6e5-802f-4d6d-8929-ab2f69dcd861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:06.573336 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934de6e5-802f-4d6d-8929-ab2f69dcd861 admin admin] PATH_INFO: `/v3/projects/5d1057cd915f42ed9744e648d7ff579a/users/5a7e8c9d48ad4266b46e9e0353c46579/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:06.575476 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934de6e5-802f-4d6d-8929-ab2f69dcd861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d1057cd915f42ed9744e648d7ff579a, user_id=5a7e8c9d48ad4266b46e9e0353c46579, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:06.578287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934de6e5-802f-4d6d-8929-ab2f69dcd861 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:06.590629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1018/2020] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:37:06 2022] PUT /identity/v3/projects/5d1057cd915f42ed9744e648d7ff579a/users/5a7e8c9d48ad4266b46e9e0353c46579/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:06.601742 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e02d5c-366b-4f4c-bf99-d7d82dcb30db None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:06.602225 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e02d5c-366b-4f4c-bf99-d7d82dcb30db None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:06.602650 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e02d5c-366b-4f4c-bf99-d7d82dcb30db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:06.634248 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-a4e02d5c-366b-4f4c-bf99-d7d82dcb30db None None] MFA Rules not processed for user `5a7e8c9d48ad4266b46e9e0353c46579`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:06.636320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-a4e02d5c-366b-4f4c-bf99-d7d82dcb30db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:06.783753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1003/2021] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:06 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:06.796268 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934de6e5-802f-4d6d-8929-ab2f69dcd861 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:06.798198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-934de6e5-802f-4d6d-8929-ab2f69dcd861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:06.811632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:06.932576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934de6e5-802f-4d6d-8929-ab2f69dcd861 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:06.938388 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931bca2a-be24-4cf4-aedd-4a92a857a942 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:06.939379 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931bca2a-be24-4cf4-aedd-4a92a857a942 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:06.939826 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931bca2a-be24-4cf4-aedd-4a92a857a942 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:06.940370 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931bca2a-be24-4cf4-aedd-4a92a857a942 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:06.956968 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931bca2a-be24-4cf4-aedd-4a92a857a942 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:06.960104 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931bca2a-be24-4cf4-aedd-4a92a857a942 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:06.982389 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1019/2022] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:37:06 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:37:07.111369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e02d5c-366b-4f4c-bf99-d7d82dcb30db None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:07.114733 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-a4e02d5c-366b-4f4c-bf99-d7d82dcb30db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:07.131233 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e02d5c-366b-4f4c-bf99-d7d82dcb30db None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:07.137099 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775cc49-d4bb-4d45-bdb5-0655b2871c7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:07.138541 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775cc49-d4bb-4d45-bdb5-0655b2871c7c service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:07.139046 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775cc49-d4bb-4d45-bdb5-0655b2871c7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:07.139473 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775cc49-d4bb-4d45-bdb5-0655b2871c7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:07.259441 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775cc49-d4bb-4d45-bdb5-0655b2871c7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:07.261744 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775cc49-d4bb-4d45-bdb5-0655b2871c7c service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:07.296493 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1004/2023] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:37:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:37:07.445774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931bca2a-be24-4cf4-aedd-4a92a857a942 service cinder] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:07.447728 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-931bca2a-be24-4cf4-aedd-4a92a857a942 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:07.532215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931bca2a-be24-4cf4-aedd-4a92a857a942 service cinder] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:07.537662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5ef0f-7651-4337-948d-e93806ea7d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:07.538323 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5ef0f-7651-4337-948d-e93806ea7d2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:07.538617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5ef0f-7651-4337-948d-e93806ea7d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:07.538885 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5ef0f-7651-4337-948d-e93806ea7d2e admin admin] PATH_INFO: `/v3/users/c5710d1f3a6742e489c7c676cc4c01d3` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:07.540087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5ef0f-7651-4337-948d-e93806ea7d2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5710d1f3a6742e489c7c676cc4c01d3)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:07.542067 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5ef0f-7651-4337-948d-e93806ea7d2e admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:07.599763 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-14d5ef0f-7651-4337-948d-e93806ea7d2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5710d1f3a6742e489c7c676cc4c01d3'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:07.611469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-14d5ef0f-7651-4337-948d-e93806ea7d2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5710d1f3a6742e489c7c676cc4c01d3'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:07.626639 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-14d5ef0f-7651-4337-948d-e93806ea7d2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5710d1f3a6742e489c7c676cc4c01d3'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:07.633659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-14d5ef0f-7651-4337-948d-e93806ea7d2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5710d1f3a6742e489c7c676cc4c01d3'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:07.634349 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-14d5ef0f-7651-4337-948d-e93806ea7d2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5710d1f3a6742e489c7c676cc4c01d3'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:07.640857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1020/2024] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:37:07 2022] DELETE /identity/v3/users/c5710d1f3a6742e489c7c676cc4c01d3 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:37:07.828852 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775cc49-d4bb-4d45-bdb5-0655b2871c7c service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:07.840988 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775cc49-d4bb-4d45-bdb5-0655b2871c7c service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:07.853184 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:07.899218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefc333-4dfc-4dcb-a0bf-5679a3a83a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:07.899865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefc333-4dfc-4dcb-a0bf-5679a3a83a0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:07.900152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefc333-4dfc-4dcb-a0bf-5679a3a83a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:07.900456 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefc333-4dfc-4dcb-a0bf-5679a3a83a0d admin admin] PATH_INFO: `/v3/projects/9f922d21e1e8434ea5e93938dc56bed9` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:07.901635 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefc333-4dfc-4dcb-a0bf-5679a3a83a0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f922d21e1e8434ea5e93938dc56bed9)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:07.903578 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefc333-4dfc-4dcb-a0bf-5679a3a83a0d admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:07.951324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-baefc333-4dfc-4dcb-a0bf-5679a3a83a0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f922d21e1e8434ea5e93938dc56bed9', 'request_id': 'req-baefc333-4dfc-4dcb-a0bf-5679a3a83a0d'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:07.957094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-baefc333-4dfc-4dcb-a0bf-5679a3a83a0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f922d21e1e8434ea5e93938dc56bed9', 'request_id': 'req-baefc333-4dfc-4dcb-a0bf-5679a3a83a0d'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:07.965814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-baefc333-4dfc-4dcb-a0bf-5679a3a83a0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f922d21e1e8434ea5e93938dc56bed9', 'request_id': 'req-baefc333-4dfc-4dcb-a0bf-5679a3a83a0d'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:07.979190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-baefc333-4dfc-4dcb-a0bf-5679a3a83a0d admin admin] The token cache is being invalidate because project 9f922d21e1e8434ea5e93938dc56bed9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:37:07.979597 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-baefc333-4dfc-4dcb-a0bf-5679a3a83a0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:07.980537 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-baefc333-4dfc-4dcb-a0bf-5679a3a83a0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:07.982351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1005/2025] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:37:07 2022] DELETE /identity/v3/projects/9f922d21e1e8434ea5e93938dc56bed9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:07.987473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3c2c5-6403-419d-ad29-07453ae1f69f None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:07.987928 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3c2c5-6403-419d-ad29-07453ae1f69f None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:07.988267 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3c2c5-6403-419d-ad29-07453ae1f69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:08.010199 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-3ec3c2c5-6403-419d-ad29-07453ae1f69f None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:08.011487 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-3ec3c2c5-6403-419d-ad29-07453ae1f69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:08.031594 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:08.074018 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1021/2026] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:08.082518 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24ee00-1e52-40ec-800e-34334b106a99 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:08.082815 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24ee00-1e52-40ec-800e-34334b106a99 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:08.083419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24ee00-1e52-40ec-800e-34334b106a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:08.105128 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-2b24ee00-1e52-40ec-800e-34334b106a99 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:08.107425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-2b24ee00-1e52-40ec-800e-34334b106a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:08.117296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1006/2027] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:37:08.126116 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3c2c5-6403-419d-ad29-07453ae1f69f None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:08.140930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3c2c5-6403-419d-ad29-07453ae1f69f None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:08.146371 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be917c02-fb6d-4d08-a355-f9bbebaf3f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:08.147805 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be917c02-fb6d-4d08-a355-f9bbebaf3f85 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:08.149010 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be917c02-fb6d-4d08-a355-f9bbebaf3f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:08.149566 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be917c02-fb6d-4d08-a355-f9bbebaf3f85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:08.150517 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be917c02-fb6d-4d08-a355-f9bbebaf3f85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:37:08.150833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be917c02-fb6d-4d08-a355-f9bbebaf3f85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:08.154540 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be917c02-fb6d-4d08-a355-f9bbebaf3f85 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:08.173472 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1022/2028] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:37:08.177457 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24ee00-1e52-40ec-800e-34334b106a99 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:08.190417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24ee00-1e52-40ec-800e-34334b106a99 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:08.195334 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c6afc-b74e-4e50-91b7-4f9311b0e136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:08.195965 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c6afc-b74e-4e50-91b7-4f9311b0e136 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:08.196249 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c6afc-b74e-4e50-91b7-4f9311b0e136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:08.196524 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c6afc-b74e-4e50-91b7-4f9311b0e136 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:08.197321 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c6afc-b74e-4e50-91b7-4f9311b0e136 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:08.199494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c6afc-b74e-4e50-91b7-4f9311b0e136 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:08.236468 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1007/2029] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:37:08 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:37:08.240274 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be917c02-fb6d-4d08-a355-f9bbebaf3f85 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:08.252836 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be917c02-fb6d-4d08-a355-f9bbebaf3f85 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:08.333758 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdd789-150c-4454-8d0f-d37fa91d9531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:08.334645 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdd789-150c-4454-8d0f-d37fa91d9531 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:08.334959 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdd789-150c-4454-8d0f-d37fa91d9531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:08.335215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdd789-150c-4454-8d0f-d37fa91d9531 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:08.335996 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdd789-150c-4454-8d0f-d37fa91d9531 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:08.338099 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdd789-150c-4454-8d0f-d37fa91d9531 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:08.362058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1023/2030] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:37:08 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:37:08.366232 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c6afc-b74e-4e50-91b7-4f9311b0e136 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:08.376986 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c6afc-b74e-4e50-91b7-4f9311b0e136 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:08.381680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924657ca-2367-46cb-b533-c2355a857b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:08.382448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924657ca-2367-46cb-b533-c2355a857b47 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:08.382830 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924657ca-2367-46cb-b533-c2355a857b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:08.383177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924657ca-2367-46cb-b533-c2355a857b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:08.383947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924657ca-2367-46cb-b533-c2355a857b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:08.385680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924657ca-2367-46cb-b533-c2355a857b47 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:08.396880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1008/2031] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:37:08.400855 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdd789-150c-4454-8d0f-d37fa91d9531 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:08.411758 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdd789-150c-4454-8d0f-d37fa91d9531 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:08.417017 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a193955-c2a7-49cf-b31f-dbaaaa73fb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:08.417737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a193955-c2a7-49cf-b31f-dbaaaa73fb64 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:08.418203 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a193955-c2a7-49cf-b31f-dbaaaa73fb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:08.418587 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a193955-c2a7-49cf-b31f-dbaaaa73fb64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:08.419399 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a193955-c2a7-49cf-b31f-dbaaaa73fb64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:08.421257 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a193955-c2a7-49cf-b31f-dbaaaa73fb64 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:08.433414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1024/2032] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:37:08.437817 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924657ca-2367-46cb-b533-c2355a857b47 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:08.447882 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924657ca-2367-46cb-b533-c2355a857b47 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:08.453270 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077f57a-0cef-439b-96bc-3cd1527b3c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:08.453928 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077f57a-0cef-439b-96bc-3cd1527b3c53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:08.454218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077f57a-0cef-439b-96bc-3cd1527b3c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:08.454498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077f57a-0cef-439b-96bc-3cd1527b3c53 admin admin] PATH_INFO: `/v3/projects/6dcf43cc1ef44b87bfe61bfb7cbd8f3a/users/7207784d76a2457189bbf2df82a17172/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:08.471745 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077f57a-0cef-439b-96bc-3cd1527b3c53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dcf43cc1ef44b87bfe61bfb7cbd8f3a, user_id=7207784d76a2457189bbf2df82a17172, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:08.474513 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077f57a-0cef-439b-96bc-3cd1527b3c53 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:08.488217 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1009/2033] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:37:08 2022] PUT /identity/v3/projects/6dcf43cc1ef44b87bfe61bfb7cbd8f3a/users/7207784d76a2457189bbf2df82a17172/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:08.499029 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2ab6e-8b34-407a-b300-9a3fe3d32c33 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:08.499319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2ab6e-8b34-407a-b300-9a3fe3d32c33 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:08.499606 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2ab6e-8b34-407a-b300-9a3fe3d32c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:08.530457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-37d2ab6e-8b34-407a-b300-9a3fe3d32c33 None None] MFA Rules not processed for user `7207784d76a2457189bbf2df82a17172`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:08.531666 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-37d2ab6e-8b34-407a-b300-9a3fe3d32c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:08.599672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1025/2034] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:08.610934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077f57a-0cef-439b-96bc-3cd1527b3c53 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:08.612824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-5077f57a-0cef-439b-96bc-3cd1527b3c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:08.660387 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:08.695716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077f57a-0cef-439b-96bc-3cd1527b3c53 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:08.703400 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68144c56-d9c0-444a-82cb-e584643bcc5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:08.703860 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68144c56-d9c0-444a-82cb-e584643bcc5a service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:08.704253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68144c56-d9c0-444a-82cb-e584643bcc5a service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:08.704635 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68144c56-d9c0-444a-82cb-e584643bcc5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:08.714742 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68144c56-d9c0-444a-82cb-e584643bcc5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:08.716509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68144c56-d9c0-444a-82cb-e584643bcc5a service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:08.726003 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1010/2035] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:37:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:37:10.508661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2ab6e-8b34-407a-b300-9a3fe3d32c33 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:10.510125 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-37d2ab6e-8b34-407a-b300-9a3fe3d32c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:10.545872 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:10.626993 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2ab6e-8b34-407a-b300-9a3fe3d32c33 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:10.631933 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a52e3-334a-413f-b446-7f3aeecb5fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:10.632666 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a52e3-334a-413f-b446-7f3aeecb5fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:10.633083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a52e3-334a-413f-b446-7f3aeecb5fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:10.633434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a52e3-334a-413f-b446-7f3aeecb5fe1 admin admin] PATH_INFO: `/v3/users/fde4b1f47fd34ac783d2874fd43e90c7` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:10.634679 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a52e3-334a-413f-b446-7f3aeecb5fe1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fde4b1f47fd34ac783d2874fd43e90c7)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:10.636617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a52e3-334a-413f-b446-7f3aeecb5fe1 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:10.698489 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-e12a52e3-334a-413f-b446-7f3aeecb5fe1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fde4b1f47fd34ac783d2874fd43e90c7'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:10.708607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-e12a52e3-334a-413f-b446-7f3aeecb5fe1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fde4b1f47fd34ac783d2874fd43e90c7'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:10.734906 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-e12a52e3-334a-413f-b446-7f3aeecb5fe1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fde4b1f47fd34ac783d2874fd43e90c7'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:10.744181 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-e12a52e3-334a-413f-b446-7f3aeecb5fe1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fde4b1f47fd34ac783d2874fd43e90c7'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:10.745056 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-e12a52e3-334a-413f-b446-7f3aeecb5fe1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fde4b1f47fd34ac783d2874fd43e90c7'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:10.752746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1026/2036] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:37:10 2022] DELETE /identity/v3/users/fde4b1f47fd34ac783d2874fd43e90c7 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:10.757202 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c48c8f-abbe-4058-8360-41151ef0287a None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:10.757655 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c48c8f-abbe-4058-8360-41151ef0287a None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:10.758399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c48c8f-abbe-4058-8360-41151ef0287a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:10.779960 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-20c48c8f-abbe-4058-8360-41151ef0287a None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:10.781197 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-20c48c8f-abbe-4058-8360-41151ef0287a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:10.852112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1011/2037] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:10.862813 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a52e3-334a-413f-b446-7f3aeecb5fe1 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:10.864330 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e12a52e3-334a-413f-b446-7f3aeecb5fe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:10.924295 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a52e3-334a-413f-b446-7f3aeecb5fe1 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:10.928238 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd4137-45a1-4801-8a07-c043bdbd9891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:10.928875 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd4137-45a1-4801-8a07-c043bdbd9891 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:10.929168 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd4137-45a1-4801-8a07-c043bdbd9891 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:10.929454 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd4137-45a1-4801-8a07-c043bdbd9891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:10.939427 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd4137-45a1-4801-8a07-c043bdbd9891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:10.940861 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd4137-45a1-4801-8a07-c043bdbd9891 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:10.951322 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1027/2038] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:37:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:37:11.250948 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c48c8f-abbe-4058-8360-41151ef0287a None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:11.252386 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-20c48c8f-abbe-4058-8360-41151ef0287a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:11.265411 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c48c8f-abbe-4058-8360-41151ef0287a None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:11.270256 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3162c-8527-47e1-b85c-ebdd3b49a85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:11.270923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3162c-8527-47e1-b85c-ebdd3b49a85b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:11.271214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3162c-8527-47e1-b85c-ebdd3b49a85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:11.271526 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3162c-8527-47e1-b85c-ebdd3b49a85b admin admin] PATH_INFO: `/v3/projects/5e5666c9be414c85a865c96036b64562` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:11.272984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3162c-8527-47e1-b85c-ebdd3b49a85b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e5666c9be414c85a865c96036b64562)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:11.275118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3162c-8527-47e1-b85c-ebdd3b49a85b admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:11.325367 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a5c3162c-8527-47e1-b85c-ebdd3b49a85b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e5666c9be414c85a865c96036b64562', 'request_id': 'req-a5c3162c-8527-47e1-b85c-ebdd3b49a85b'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:11.331170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a5c3162c-8527-47e1-b85c-ebdd3b49a85b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e5666c9be414c85a865c96036b64562', 'request_id': 'req-a5c3162c-8527-47e1-b85c-ebdd3b49a85b'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:11.339035 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a5c3162c-8527-47e1-b85c-ebdd3b49a85b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e5666c9be414c85a865c96036b64562', 'request_id': 'req-a5c3162c-8527-47e1-b85c-ebdd3b49a85b'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:11.350310 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a5c3162c-8527-47e1-b85c-ebdd3b49a85b admin admin] The token cache is being invalidate because project 5e5666c9be414c85a865c96036b64562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:37:11.350594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a5c3162c-8527-47e1-b85c-ebdd3b49a85b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:11.351232 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a5c3162c-8527-47e1-b85c-ebdd3b49a85b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:11.352735 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1012/2039] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:37:11 2022] DELETE /identity/v3/projects/5e5666c9be414c85a865c96036b64562 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:11.357535 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd4137-45a1-4801-8a07-c043bdbd9891 service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:11.368151 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd4137-45a1-4801-8a07-c043bdbd9891 service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:11.418359 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:11.445193 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593e3f6-fd11-475e-9969-ccce09d768a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:11.445193 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593e3f6-fd11-475e-9969-ccce09d768a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:11.445193 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593e3f6-fd11-475e-9969-ccce09d768a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:11.445193 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593e3f6-fd11-475e-9969-ccce09d768a5 admin admin] PATH_INFO: `/v3/users/08d670fd660c43c4b55addc5b1f636a1` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:11.446550 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593e3f6-fd11-475e-9969-ccce09d768a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08d670fd660c43c4b55addc5b1f636a1)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:11.446699 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3162c-8527-47e1-b85c-ebdd3b49a85b admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:11.447931 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-a5c3162c-8527-47e1-b85c-ebdd3b49a85b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:11.448670 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593e3f6-fd11-475e-9969-ccce09d768a5 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:11.461566 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3162c-8527-47e1-b85c-ebdd3b49a85b admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:11.467009 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fc5a7-218f-4a7c-a890-26179c13e140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:11.467652 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fc5a7-218f-4a7c-a890-26179c13e140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:11.467932 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fc5a7-218f-4a7c-a890-26179c13e140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:11.468204 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fc5a7-218f-4a7c-a890-26179c13e140 admin admin] PATH_INFO: `/v3/users/7207784d76a2457189bbf2df82a17172` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:11.469643 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165fc5a7-218f-4a7c-a890-26179c13e140 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7207784d76a2457189bbf2df82a17172)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:11.471629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165fc5a7-218f-4a7c-a890-26179c13e140 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:11.509692 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-e593e3f6-fd11-475e-9969-ccce09d768a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08d670fd660c43c4b55addc5b1f636a1'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:11.518021 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-e593e3f6-fd11-475e-9969-ccce09d768a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08d670fd660c43c4b55addc5b1f636a1'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:11.527324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:11.529723 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-165fc5a7-218f-4a7c-a890-26179c13e140 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7207784d76a2457189bbf2df82a17172'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:11.532484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-e593e3f6-fd11-475e-9969-ccce09d768a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08d670fd660c43c4b55addc5b1f636a1'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:11.539818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-165fc5a7-218f-4a7c-a890-26179c13e140 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7207784d76a2457189bbf2df82a17172'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:11.540719 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-e593e3f6-fd11-475e-9969-ccce09d768a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08d670fd660c43c4b55addc5b1f636a1'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:11.541433 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-e593e3f6-fd11-475e-9969-ccce09d768a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08d670fd660c43c4b55addc5b1f636a1'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:11.547764 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-165fc5a7-218f-4a7c-a890-26179c13e140 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7207784d76a2457189bbf2df82a17172'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:11.548128 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1028/2040] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:37:11 2022] DELETE /identity/v3/users/08d670fd660c43c4b55addc5b1f636a1 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:11.562186 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-165fc5a7-218f-4a7c-a890-26179c13e140 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7207784d76a2457189bbf2df82a17172'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:11.563008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-165fc5a7-218f-4a7c-a890-26179c13e140 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7207784d76a2457189bbf2df82a17172'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:11.570463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1013/2041] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:37:11 2022] DELETE /identity/v3/users/7207784d76a2457189bbf2df82a17172 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:37:11.575151 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916ca9c-a15f-4f9d-bde5-dd090ff59cd3 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:11.575493 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916ca9c-a15f-4f9d-bde5-dd090ff59cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:11.575726 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916ca9c-a15f-4f9d-bde5-dd090ff59cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:11.598136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-b916ca9c-a15f-4f9d-bde5-dd090ff59cd3 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:11.599361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b916ca9c-a15f-4f9d-bde5-dd090ff59cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:11.691184 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1029/2042] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:11.706950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fc5a7-218f-4a7c-a890-26179c13e140 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:11.708499 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-165fc5a7-218f-4a7c-a890-26179c13e140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:11.814199 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fc5a7-218f-4a7c-a890-26179c13e140 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:11.818122 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217636e-494f-4a3e-871c-1aa14bd23e74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:11.818744 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217636e-494f-4a3e-871c-1aa14bd23e74 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:11.819017 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217636e-494f-4a3e-871c-1aa14bd23e74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:11.819449 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217636e-494f-4a3e-871c-1aa14bd23e74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:11.829893 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f217636e-494f-4a3e-871c-1aa14bd23e74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:11.831338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f217636e-494f-4a3e-871c-1aa14bd23e74 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:11.851579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1014/2043] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:37:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:37:12.063402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916ca9c-a15f-4f9d-bde5-dd090ff59cd3 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:12.065220 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b916ca9c-a15f-4f9d-bde5-dd090ff59cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:12.081581 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916ca9c-a15f-4f9d-bde5-dd090ff59cd3 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:12.091634 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25673d7f-7eb1-4406-b25d-66ef39e7fb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:12.108213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25673d7f-7eb1-4406-b25d-66ef39e7fb6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:12.109459 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25673d7f-7eb1-4406-b25d-66ef39e7fb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:12.109459 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25673d7f-7eb1-4406-b25d-66ef39e7fb6d admin admin] PATH_INFO: `/v3/projects/c74ca0fbb5a1491db93cd252a975e6b6` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:12.113753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25673d7f-7eb1-4406-b25d-66ef39e7fb6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c74ca0fbb5a1491db93cd252a975e6b6)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:12.116799 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25673d7f-7eb1-4406-b25d-66ef39e7fb6d admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:12.191950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-25673d7f-7eb1-4406-b25d-66ef39e7fb6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c74ca0fbb5a1491db93cd252a975e6b6', 'request_id': 'req-25673d7f-7eb1-4406-b25d-66ef39e7fb6d'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:12.205379 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-25673d7f-7eb1-4406-b25d-66ef39e7fb6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c74ca0fbb5a1491db93cd252a975e6b6', 'request_id': 'req-25673d7f-7eb1-4406-b25d-66ef39e7fb6d'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:12.205379 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:12.213200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-25673d7f-7eb1-4406-b25d-66ef39e7fb6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c74ca0fbb5a1491db93cd252a975e6b6', 'request_id': 'req-25673d7f-7eb1-4406-b25d-66ef39e7fb6d'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:12.227565 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-25673d7f-7eb1-4406-b25d-66ef39e7fb6d admin admin] The token cache is being invalidate because project c74ca0fbb5a1491db93cd252a975e6b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:37:12.227821 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-25673d7f-7eb1-4406-b25d-66ef39e7fb6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:12.228632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-25673d7f-7eb1-4406-b25d-66ef39e7fb6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:12.230988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1030/2044] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:37:12 2022] DELETE /identity/v3/projects/c74ca0fbb5a1491db93cd252a975e6b6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:12.236926 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04974faf-05a7-4adc-bc4d-26cc462d1a5c None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:12.237245 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04974faf-05a7-4adc-bc4d-26cc462d1a5c None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:12.237546 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04974faf-05a7-4adc-bc4d-26cc462d1a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:12.262445 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-04974faf-05a7-4adc-bc4d-26cc462d1a5c None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:12.264215 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-04974faf-05a7-4adc-bc4d-26cc462d1a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:12.293691 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25673d7f-7eb1-4406-b25d-66ef39e7fb6d admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:12.296085 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-25673d7f-7eb1-4406-b25d-66ef39e7fb6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:12.339629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1015/2045] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:12.347770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a9a77-8e4b-4719-b506-1cefdf2415c4 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:12.348067 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a9a77-8e4b-4719-b506-1cefdf2415c4 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:12.348518 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a9a77-8e4b-4719-b506-1cefdf2415c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:12.368341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-396a9a77-8e4b-4719-b506-1cefdf2415c4 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:12.369647 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-396a9a77-8e4b-4719-b506-1cefdf2415c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:12.372819 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25673d7f-7eb1-4406-b25d-66ef39e7fb6d admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:12.377568 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91fae2-d779-4506-95c1-09968ced2628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:12.378223 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91fae2-d779-4506-95c1-09968ced2628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:12.378485 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91fae2-d779-4506-95c1-09968ced2628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:12.378815 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91fae2-d779-4506-95c1-09968ced2628 admin admin] PATH_INFO: `/v3/users/487b854cb9ba4fc2ad732061f6772b5b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:12.380044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1016/2046] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:37:12.380381 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91fae2-d779-4506-95c1-09968ced2628 admin admin] RBAC: Authorizing `identity:delete_user(user_id=487b854cb9ba4fc2ad732061f6772b5b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:12.382360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91fae2-d779-4506-95c1-09968ced2628 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:12.384188 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a9a77-8e4b-4719-b506-1cefdf2415c4 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:12.394351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a9a77-8e4b-4719-b506-1cefdf2415c4 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:12.399404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0001a34-f128-4634-91c4-494056236822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:12.400005 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0001a34-f128-4634-91c4-494056236822 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:12.400346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0001a34-f128-4634-91c4-494056236822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:12.400642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0001a34-f128-4634-91c4-494056236822 admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:12.401381 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0001a34-f128-4634-91c4-494056236822 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:37:12.401751 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0001a34-f128-4634-91c4-494056236822 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:12.403328 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0001a34-f128-4634-91c4-494056236822 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:12.415971 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1017/2047] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:37:12.420507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0001a34-f128-4634-91c4-494056236822 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:12.422385 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-a0001a34-f128-4634-91c4-494056236822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:12.438808 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0001a34-f128-4634-91c4-494056236822 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:12.443849 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be330c-a11c-4e02-94b6-13cc8478c0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:12.444984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be330c-a11c-4e02-94b6-13cc8478c0c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:12.445293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be330c-a11c-4e02-94b6-13cc8478c0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:12.445604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be330c-a11c-4e02-94b6-13cc8478c0c3 admin admin] PATH_INFO: `/v3/projects/6dcf43cc1ef44b87bfe61bfb7cbd8f3a` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:12.445736 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-7b91fae2-d779-4506-95c1-09968ced2628 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '487b854cb9ba4fc2ad732061f6772b5b'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:12.446812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be330c-a11c-4e02-94b6-13cc8478c0c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dcf43cc1ef44b87bfe61bfb7cbd8f3a)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:12.448858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be330c-a11c-4e02-94b6-13cc8478c0c3 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:12.454487 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-7b91fae2-d779-4506-95c1-09968ced2628 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '487b854cb9ba4fc2ad732061f6772b5b'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:12.467196 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-7b91fae2-d779-4506-95c1-09968ced2628 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '487b854cb9ba4fc2ad732061f6772b5b'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:12.474496 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-7b91fae2-d779-4506-95c1-09968ced2628 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '487b854cb9ba4fc2ad732061f6772b5b'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:12.475181 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-7b91fae2-d779-4506-95c1-09968ced2628 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '487b854cb9ba4fc2ad732061f6772b5b'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:12.481781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1031/2048] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:37:12 2022] DELETE /identity/v3/users/487b854cb9ba4fc2ad732061f6772b5b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:12.485447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91fae2-d779-4506-95c1-09968ced2628 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:12.486792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7b91fae2-d779-4506-95c1-09968ced2628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:12.491675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:12.499293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-c2be330c-a11c-4e02-94b6-13cc8478c0c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dcf43cc1ef44b87bfe61bfb7cbd8f3a', 'request_id': 'req-c2be330c-a11c-4e02-94b6-13cc8478c0c3'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:12.505950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-c2be330c-a11c-4e02-94b6-13cc8478c0c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dcf43cc1ef44b87bfe61bfb7cbd8f3a', 'request_id': 'req-c2be330c-a11c-4e02-94b6-13cc8478c0c3'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:12.514211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-c2be330c-a11c-4e02-94b6-13cc8478c0c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dcf43cc1ef44b87bfe61bfb7cbd8f3a', 'request_id': 'req-c2be330c-a11c-4e02-94b6-13cc8478c0c3'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:12.526071 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-c2be330c-a11c-4e02-94b6-13cc8478c0c3 admin admin] The token cache is being invalidate because project 6dcf43cc1ef44b87bfe61bfb7cbd8f3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:37:12.526363 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-c2be330c-a11c-4e02-94b6-13cc8478c0c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:12.527016 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-c2be330c-a11c-4e02-94b6-13cc8478c0c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:12.528582 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1018/2049] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:37:12 2022] DELETE /identity/v3/projects/6dcf43cc1ef44b87bfe61bfb7cbd8f3a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:12.532255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c8884-1c98-4c29-8bd4-6e7f903645d5 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:12.532255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c8884-1c98-4c29-8bd4-6e7f903645d5 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:12.532255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c8884-1c98-4c29-8bd4-6e7f903645d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:12.549529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-f99c8884-1c98-4c29-8bd4-6e7f903645d5 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:12.550730 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-f99c8884-1c98-4c29-8bd4-6e7f903645d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:12.553992 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91fae2-d779-4506-95c1-09968ced2628 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:12.562783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:12.613636 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1019/2050] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:12.615050 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb8609-7d95-4159-b247-8e431fd0eb19 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:12.615159 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de54c1-c6e1-4682-b92b-9bbfc481d07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:12.615572 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb8609-7d95-4159-b247-8e431fd0eb19 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:12.615674 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de54c1-c6e1-4682-b92b-9bbfc481d07e admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:12.616070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de54c1-c6e1-4682-b92b-9bbfc481d07e admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:12.616331 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb8609-7d95-4159-b247-8e431fd0eb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:12.616427 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de54c1-c6e1-4682-b92b-9bbfc481d07e admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:12.617231 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de54c1-c6e1-4682-b92b-9bbfc481d07e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:12.619234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de54c1-c6e1-4682-b92b-9bbfc481d07e admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:12.636020 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-0ffb8609-7d95-4159-b247-8e431fd0eb19 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:12.638041 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0ffb8609-7d95-4159-b247-8e431fd0eb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:12.646583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1020/2051] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:12.647353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb8609-7d95-4159-b247-8e431fd0eb19 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:12.647871 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1032/2052] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:37:12 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:37:12.648714 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0ffb8609-7d95-4159-b247-8e431fd0eb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:12.656302 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c0335-c2f9-4085-b077-2d50632506c8 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:12.656575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c0335-c2f9-4085-b077-2d50632506c8 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:12.656855 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c0335-c2f9-4085-b077-2d50632506c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:12.675893 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-820c0335-c2f9-4085-b077-2d50632506c8 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:12.677119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-820c0335-c2f9-4085-b077-2d50632506c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:12.710312 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb8609-7d95-4159-b247-8e431fd0eb19 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:12.741079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1033/2053] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:37:12.741900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c0335-c2f9-4085-b077-2d50632506c8 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:12.743195 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-820c0335-c2f9-4085-b077-2d50632506c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:12.755199 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c0335-c2f9-4085-b077-2d50632506c8 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:12.759691 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6066330c-5aa1-43a6-a3a7-ed3fed0c6121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:12.760289 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066330c-5aa1-43a6-a3a7-ed3fed0c6121 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:12.760572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066330c-5aa1-43a6-a3a7-ed3fed0c6121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:12.760869 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6066330c-5aa1-43a6-a3a7-ed3fed0c6121 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:12.761558 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6066330c-5aa1-43a6-a3a7-ed3fed0c6121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:12.764025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6066330c-5aa1-43a6-a3a7-ed3fed0c6121 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:12.764474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200a437-04c1-4b42-9f1d-bed4def4de55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:12.765145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200a437-04c1-4b42-9f1d-bed4def4de55 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:12.765429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200a437-04c1-4b42-9f1d-bed4def4de55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:12.765701 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200a437-04c1-4b42-9f1d-bed4def4de55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:12.774917 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200a437-04c1-4b42-9f1d-bed4def4de55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:12.776485 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200a437-04c1-4b42-9f1d-bed4def4de55 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:12.789745 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1034/2054] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:37:12 2022] POST /identity/v3/users => generated 293 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:37:12.789932 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1021/2055] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:37:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:37:12.790038 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6066330c-5aa1-43a6-a3a7-ed3fed0c6121 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:12.790741 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200a437-04c1-4b42-9f1d-bed4def4de55 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:12.800483 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6066330c-5aa1-43a6-a3a7-ed3fed0c6121 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:12.801465 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200a437-04c1-4b42-9f1d-bed4def4de55 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:12.806167 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676a785-d9af-44cf-b0dc-a16a846b6bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:12.806867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676a785-d9af-44cf-b0dc-a16a846b6bab admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:12.807224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676a785-d9af-44cf-b0dc-a16a846b6bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:12.807455 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f446c-f21c-43cd-b858-ea52fa28f548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:12.807645 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676a785-d9af-44cf-b0dc-a16a846b6bab admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:12.808195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f446c-f21c-43cd-b858-ea52fa28f548 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:12.808544 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676a785-d9af-44cf-b0dc-a16a846b6bab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:37:12.808658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f446c-f21c-43cd-b858-ea52fa28f548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:12.808984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f446c-f21c-43cd-b858-ea52fa28f548 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:12.809096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676a785-d9af-44cf-b0dc-a16a846b6bab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:12.809849 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f446c-f21c-43cd-b858-ea52fa28f548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:12.810946 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676a785-d9af-44cf-b0dc-a16a846b6bab admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:12.811570 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f446c-f21c-43cd-b858-ea52fa28f548 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:12.825854 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1022/2056] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:37:12.827211 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f446c-f21c-43cd-b858-ea52fa28f548 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:12.834287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1035/2057] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:37:12.835490 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676a785-d9af-44cf-b0dc-a16a846b6bab admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:12.855446 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f446c-f21c-43cd-b858-ea52fa28f548 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:12.855446 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa4ad0a-9619-4a2b-ae26-0b4e82ec4488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:12.856237 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4ad0a-9619-4a2b-ae26-0b4e82ec4488 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:12.856706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4ad0a-9619-4a2b-ae26-0b4e82ec4488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:12.857369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4ad0a-9619-4a2b-ae26-0b4e82ec4488 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:12.858626 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa4ad0a-9619-4a2b-ae26-0b4e82ec4488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:12.863308 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa4ad0a-9619-4a2b-ae26-0b4e82ec4488 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:12.871514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:12.874722 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676a785-d9af-44cf-b0dc-a16a846b6bab admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:12.881066 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b2060-fc80-468a-b43f-94ffbb019996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:12.884957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1023/2058] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:37:12.885470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b2060-fc80-468a-b43f-94ffbb019996 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:12.886238 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b2060-fc80-468a-b43f-94ffbb019996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:12.886710 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b2060-fc80-468a-b43f-94ffbb019996 admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:12.887398 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa4ad0a-9619-4a2b-ae26-0b4e82ec4488 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:12.888476 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b2060-fc80-468a-b43f-94ffbb019996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:12.892722 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b2060-fc80-468a-b43f-94ffbb019996 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:12.906275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa4ad0a-9619-4a2b-ae26-0b4e82ec4488 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:12.913425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612cb15-0829-4a7d-b7d4-90a82a6d0e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:12.914430 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612cb15-0829-4a7d-b7d4-90a82a6d0e3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:12.914889 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612cb15-0829-4a7d-b7d4-90a82a6d0e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:12.915342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612cb15-0829-4a7d-b7d4-90a82a6d0e3e admin admin] PATH_INFO: `/v3/projects/4f47d0ec4624485ab891405760ac8823/users/0ba6171641bd4e579e4690aeff9a38a3/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:12.939406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1036/2059] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:37:12 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:37:12.939632 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d612cb15-0829-4a7d-b7d4-90a82a6d0e3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f47d0ec4624485ab891405760ac8823, user_id=0ba6171641bd4e579e4690aeff9a38a3, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:12.941019 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b2060-fc80-468a-b43f-94ffbb019996 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:12.944046 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d612cb15-0829-4a7d-b7d4-90a82a6d0e3e admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:12.977663 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1024/2060] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:37:12 2022] PUT /identity/v3/projects/4f47d0ec4624485ab891405760ac8823/users/0ba6171641bd4e579e4690aeff9a38a3/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:12.984675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3b1b8-9e66-438d-b242-edd425c3508f None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:12.985362 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3b1b8-9e66-438d-b242-edd425c3508f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:12.985797 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3b1b8-9e66-438d-b242-edd425c3508f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:13.018185 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-96b3b1b8-9e66-438d-b242-edd425c3508f None None] MFA Rules not processed for user `0ba6171641bd4e579e4690aeff9a38a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:13.018453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:13.019531 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-96b3b1b8-9e66-438d-b242-edd425c3508f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:13.042539 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b2060-fc80-468a-b43f-94ffbb019996 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:13.102320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1025/2061] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:13.108667 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65591e0-2fca-4d64-9f4d-d1e89fc4f895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:13.108667 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65591e0-2fca-4d64-9f4d-d1e89fc4f895 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:13.108978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65591e0-2fca-4d64-9f4d-d1e89fc4f895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:13.109259 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65591e0-2fca-4d64-9f4d-d1e89fc4f895 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:13.109937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65591e0-2fca-4d64-9f4d-d1e89fc4f895 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:13.113853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65591e0-2fca-4d64-9f4d-d1e89fc4f895 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:13.117153 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3b1b8-9e66-438d-b242-edd425c3508f None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:13.118968 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-96b3b1b8-9e66-438d-b242-edd425c3508f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:13.140712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1037/2062] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:37:12 2022] POST /identity/v3/users => generated 287 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:37:13.144697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65591e0-2fca-4d64-9f4d-d1e89fc4f895 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:13.160856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65591e0-2fca-4d64-9f4d-d1e89fc4f895 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:13.167982 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ad028-722e-41e8-82c1-6bc7d65952cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:13.171099 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ad028-722e-41e8-82c1-6bc7d65952cd admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:13.171554 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ad028-722e-41e8-82c1-6bc7d65952cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:13.172009 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ad028-722e-41e8-82c1-6bc7d65952cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:13.173177 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ad028-722e-41e8-82c1-6bc7d65952cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:13.175832 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ad028-722e-41e8-82c1-6bc7d65952cd admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:13.189379 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1038/2063] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:37:13.193446 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ad028-722e-41e8-82c1-6bc7d65952cd admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:13.193667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3b1b8-9e66-438d-b242-edd425c3508f None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:13.197779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5123e6-5cd9-4f77-95a6-18202a8840bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:13.198517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5123e6-5cd9-4f77-95a6-18202a8840bb service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:13.198804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5123e6-5cd9-4f77-95a6-18202a8840bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:13.199083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5123e6-5cd9-4f77-95a6-18202a8840bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:13.210890 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5123e6-5cd9-4f77-95a6-18202a8840bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:13.212468 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5123e6-5cd9-4f77-95a6-18202a8840bb service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:13.213971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ad028-722e-41e8-82c1-6bc7d65952cd admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:13.221136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ee9c7-a3e6-4948-bfdb-f0aaa268cbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:13.221775 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ee9c7-a3e6-4948-bfdb-f0aaa268cbe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:13.222520 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ee9c7-a3e6-4948-bfdb-f0aaa268cbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:13.222807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ee9c7-a3e6-4948-bfdb-f0aaa268cbe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:13.223853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64ee9c7-a3e6-4948-bfdb-f0aaa268cbe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:13.224575 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1026/2064] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:37:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:37:13.226393 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64ee9c7-a3e6-4948-bfdb-f0aaa268cbe3 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:13.241165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1039/2065] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:37:13.245190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5123e6-5cd9-4f77-95a6-18202a8840bb service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:13.256656 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ee9c7-a3e6-4948-bfdb-f0aaa268cbe3 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:13.266622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5123e6-5cd9-4f77-95a6-18202a8840bb service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:13.274679 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ee9c7-a3e6-4948-bfdb-f0aaa268cbe3 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:13.277353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392dfdd-94dc-419f-8a10-3357676b8eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:13.278028 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392dfdd-94dc-419f-8a10-3357676b8eb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:13.278344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392dfdd-94dc-419f-8a10-3357676b8eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:13.278607 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392dfdd-94dc-419f-8a10-3357676b8eb8 admin admin] PATH_INFO: `/v3/projects/cd9632a21e414bb18333374c294af0eb/users/90a59c8caadb4242b7fca6c146a8cd20/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:13.291491 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236273f0-1a0c-4cf5-98da-7079f0dbf542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:13.292148 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236273f0-1a0c-4cf5-98da-7079f0dbf542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:13.292436 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236273f0-1a0c-4cf5-98da-7079f0dbf542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:13.292699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236273f0-1a0c-4cf5-98da-7079f0dbf542 admin admin] PATH_INFO: `/v3/users/0ba6171641bd4e579e4690aeff9a38a3` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:13.296004 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236273f0-1a0c-4cf5-98da-7079f0dbf542 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ba6171641bd4e579e4690aeff9a38a3)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:13.298837 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236273f0-1a0c-4cf5-98da-7079f0dbf542 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:13.302365 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392dfdd-94dc-419f-8a10-3357676b8eb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd9632a21e414bb18333374c294af0eb, user_id=90a59c8caadb4242b7fca6c146a8cd20, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:13.305479 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392dfdd-94dc-419f-8a10-3357676b8eb8 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:13.337030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1027/2066] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:37:13 2022] PUT /identity/v3/projects/cd9632a21e414bb18333374c294af0eb/users/90a59c8caadb4242b7fca6c146a8cd20/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:13.337030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392dfdd-94dc-419f-8a10-3357676b8eb8 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:13.341255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b392dfdd-94dc-419f-8a10-3357676b8eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:13.373767 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-236273f0-1a0c-4cf5-98da-7079f0dbf542 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ba6171641bd4e579e4690aeff9a38a3'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:13.385369 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-236273f0-1a0c-4cf5-98da-7079f0dbf542 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ba6171641bd4e579e4690aeff9a38a3'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:13.402745 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-236273f0-1a0c-4cf5-98da-7079f0dbf542 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ba6171641bd4e579e4690aeff9a38a3'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:13.409626 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:13.416857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-236273f0-1a0c-4cf5-98da-7079f0dbf542 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ba6171641bd4e579e4690aeff9a38a3'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:13.418166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-236273f0-1a0c-4cf5-98da-7079f0dbf542 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ba6171641bd4e579e4690aeff9a38a3'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:13.427952 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1040/2067] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:37:13 2022] DELETE /identity/v3/users/0ba6171641bd4e579e4690aeff9a38a3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:13.430269 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d98044-a0a7-4999-a979-2e9f9ec76b61 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:13.430569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d98044-a0a7-4999-a979-2e9f9ec76b61 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:13.431161 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d98044-a0a7-4999-a979-2e9f9ec76b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:13.444696 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392dfdd-94dc-419f-8a10-3357676b8eb8 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:13.466158 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-f8d98044-a0a7-4999-a979-2e9f9ec76b61 None None] MFA Rules not processed for user `90a59c8caadb4242b7fca6c146a8cd20`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:13.467468 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f8d98044-a0a7-4999-a979-2e9f9ec76b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:13.486953 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:13.514965 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54659e-0d91-41e9-b45f-2a373fa49ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:13.515724 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54659e-0d91-41e9-b45f-2a373fa49ab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:13.516263 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54659e-0d91-41e9-b45f-2a373fa49ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:13.516263 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54659e-0d91-41e9-b45f-2a373fa49ab1 admin admin] PATH_INFO: `/v3/projects/113def71e1724f5e8a776a63bca122f1` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:13.517519 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54659e-0d91-41e9-b45f-2a373fa49ab1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=113def71e1724f5e8a776a63bca122f1)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:13.519690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54659e-0d91-41e9-b45f-2a373fa49ab1 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:13.535149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1041/2068] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:13.539830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79953281-afcd-404f-9e43-3783edb2fce4 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:13.540115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79953281-afcd-404f-9e43-3783edb2fce4 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:13.540389 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79953281-afcd-404f-9e43-3783edb2fce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:13.558906 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-79953281-afcd-404f-9e43-3783edb2fce4 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:13.560770 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-79953281-afcd-404f-9e43-3783edb2fce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:13.568503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-db54659e-0d91-41e9-b45f-2a373fa49ab1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '113def71e1724f5e8a776a63bca122f1', 'request_id': 'req-db54659e-0d91-41e9-b45f-2a373fa49ab1'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:13.574270 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-db54659e-0d91-41e9-b45f-2a373fa49ab1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '113def71e1724f5e8a776a63bca122f1', 'request_id': 'req-db54659e-0d91-41e9-b45f-2a373fa49ab1'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:13.581360 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-db54659e-0d91-41e9-b45f-2a373fa49ab1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '113def71e1724f5e8a776a63bca122f1', 'request_id': 'req-db54659e-0d91-41e9-b45f-2a373fa49ab1'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:13.592443 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-db54659e-0d91-41e9-b45f-2a373fa49ab1 admin admin] The token cache is being invalidate because project 113def71e1724f5e8a776a63bca122f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:37:13.592747 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-db54659e-0d91-41e9-b45f-2a373fa49ab1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:13.593401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-db54659e-0d91-41e9-b45f-2a373fa49ab1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:13.594912 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1028/2069] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:37:13 2022] DELETE /identity/v3/projects/113def71e1724f5e8a776a63bca122f1 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:13.595646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54659e-0d91-41e9-b45f-2a373fa49ab1 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:13.596961 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-db54659e-0d91-41e9-b45f-2a373fa49ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:13.628587 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1042/2070] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:13.630538 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79953281-afcd-404f-9e43-3783edb2fce4 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:13.632239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-79953281-afcd-404f-9e43-3783edb2fce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:13.643902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79953281-afcd-404f-9e43-3783edb2fce4 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:13.648538 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f8701-94ed-465d-997e-a4bc49684ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:13.649223 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f8701-94ed-465d-997e-a4bc49684ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:13.649500 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f8701-94ed-465d-997e-a4bc49684ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:13.649776 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f8701-94ed-465d-997e-a4bc49684ed1 admin admin] PATH_INFO: `/v3/users/5a7e8c9d48ad4266b46e9e0353c46579` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:13.650930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f8701-94ed-465d-997e-a4bc49684ed1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a7e8c9d48ad4266b46e9e0353c46579)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:13.652845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f8701-94ed-465d-997e-a4bc49684ed1 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:13.670761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54659e-0d91-41e9-b45f-2a373fa49ab1 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:13.676228 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f92d32-190f-4140-bd3d-c19533728c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:13.676868 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f92d32-190f-4140-bd3d-c19533728c70 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:13.677145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f92d32-190f-4140-bd3d-c19533728c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:13.677424 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f92d32-190f-4140-bd3d-c19533728c70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:13.678103 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f92d32-190f-4140-bd3d-c19533728c70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:13.679978 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f92d32-190f-4140-bd3d-c19533728c70 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:13.699357 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-d60f8701-94ed-465d-997e-a4bc49684ed1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a7e8c9d48ad4266b46e9e0353c46579'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:13.706472 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-d60f8701-94ed-465d-997e-a4bc49684ed1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a7e8c9d48ad4266b46e9e0353c46579'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:13.711272 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1029/2071] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:37:13 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:37:13.711272 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f92d32-190f-4140-bd3d-c19533728c70 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:13.711272 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c5f92d32-190f-4140-bd3d-c19533728c70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:13.720461 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:13.722540 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-d60f8701-94ed-465d-997e-a4bc49684ed1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a7e8c9d48ad4266b46e9e0353c46579'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:13.730744 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-d60f8701-94ed-465d-997e-a4bc49684ed1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a7e8c9d48ad4266b46e9e0353c46579'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:13.731984 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-d60f8701-94ed-465d-997e-a4bc49684ed1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a7e8c9d48ad4266b46e9e0353c46579'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:13.738093 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1043/2072] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:37:13 2022] DELETE /identity/v3/users/5a7e8c9d48ad4266b46e9e0353c46579 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:13.739480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f8701-94ed-465d-997e-a4bc49684ed1 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:13.741052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d60f8701-94ed-465d-997e-a4bc49684ed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:13.776277 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f92d32-190f-4140-bd3d-c19533728c70 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:13.785020 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:13.824501 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f8701-94ed-465d-997e-a4bc49684ed1 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:13.831950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fce53d-f10e-42b3-81c2-5b1fc1583f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:13.832637 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fce53d-f10e-42b3-81c2-5b1fc1583f44 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:13.833010 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fce53d-f10e-42b3-81c2-5b1fc1583f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:13.833250 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fce53d-f10e-42b3-81c2-5b1fc1583f44 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:13.834078 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fce53d-f10e-42b3-81c2-5b1fc1583f44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:13.836273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fce53d-f10e-42b3-81c2-5b1fc1583f44 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:13.837086 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37ec15-ae67-4506-a3a1-5dd25f3b5953 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:13.838030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37ec15-ae67-4506-a3a1-5dd25f3b5953 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:13.838532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37ec15-ae67-4506-a3a1-5dd25f3b5953 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:13.839595 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37ec15-ae67-4506-a3a1-5dd25f3b5953 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:13.842516 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-fd37ec15-ae67-4506-a3a1-5dd25f3b5953 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:13.861953 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37ec15-ae67-4506-a3a1-5dd25f3b5953 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:13.863793 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37ec15-ae67-4506-a3a1-5dd25f3b5953 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:13.873162 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1044/2073] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:37:13 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:37:13.874212 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fce53d-f10e-42b3-81c2-5b1fc1583f44 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:13.883804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1030/2074] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:37:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:37:13.887858 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fce53d-f10e-42b3-81c2-5b1fc1583f44 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:13.893184 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ce1e8-22a8-4574-ae40-687f7204cd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:13.893897 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ce1e8-22a8-4574-ae40-687f7204cd64 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:13.894207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ce1e8-22a8-4574-ae40-687f7204cd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:13.894480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ce1e8-22a8-4574-ae40-687f7204cd64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:13.895184 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ce1e8-22a8-4574-ae40-687f7204cd64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:13.896849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ce1e8-22a8-4574-ae40-687f7204cd64 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:13.911460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1045/2075] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:37:13.917312 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37ec15-ae67-4506-a3a1-5dd25f3b5953 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:13.930443 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37ec15-ae67-4506-a3a1-5dd25f3b5953 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:13.936291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91864a-82fb-4662-ad08-48a50fecb4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:13.937594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91864a-82fb-4662-ad08-48a50fecb4a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:13.937594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91864a-82fb-4662-ad08-48a50fecb4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:13.937594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91864a-82fb-4662-ad08-48a50fecb4a8 admin admin] PATH_INFO: `/v3/projects/0540193d30f14c418991f3c2e0465670/users/a2013e1e0b674a90889c1f4462aeff75/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:13.955100 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91864a-82fb-4662-ad08-48a50fecb4a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0540193d30f14c418991f3c2e0465670, user_id=a2013e1e0b674a90889c1f4462aeff75, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:13.957871 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91864a-82fb-4662-ad08-48a50fecb4a8 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:13.976920 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1031/2076] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:37:13 2022] PUT /identity/v3/projects/0540193d30f14c418991f3c2e0465670/users/a2013e1e0b674a90889c1f4462aeff75/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:37:13.982009 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ce1e8-22a8-4574-ae40-687f7204cd64 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:13.992428 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ce1e8-22a8-4574-ae40-687f7204cd64 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:14.058348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43203a8-df7f-412d-aaba-867efd41846f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:14.059455 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43203a8-df7f-412d-aaba-867efd41846f admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:14.059455 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43203a8-df7f-412d-aaba-867efd41846f admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:14.059675 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43203a8-df7f-412d-aaba-867efd41846f admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:14.060516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43203a8-df7f-412d-aaba-867efd41846f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:14.062408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43203a8-df7f-412d-aaba-867efd41846f admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:14.074119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1046/2077] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:37:14.078453 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91864a-82fb-4662-ad08-48a50fecb4a8 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:14.091218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91864a-82fb-4662-ad08-48a50fecb4a8 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:14.096951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125faca-d08b-4026-8183-14cc7e2b34b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:14.098113 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125faca-d08b-4026-8183-14cc7e2b34b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:14.098466 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125faca-d08b-4026-8183-14cc7e2b34b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:14.098728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125faca-d08b-4026-8183-14cc7e2b34b7 admin admin] PATH_INFO: `/v3/projects/0540193d30f14c418991f3c2e0465670/users/a2013e1e0b674a90889c1f4462aeff75/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:14.101467 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0125faca-d08b-4026-8183-14cc7e2b34b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0540193d30f14c418991f3c2e0465670, user_id=a2013e1e0b674a90889c1f4462aeff75, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:14.104453 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0125faca-d08b-4026-8183-14cc7e2b34b7 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:14.116567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1032/2078] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:37:14 2022] PUT /identity/v3/projects/0540193d30f14c418991f3c2e0465670/users/a2013e1e0b674a90889c1f4462aeff75/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:37:14.127336 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cd3a9-75f3-467a-926b-ca753f4c8141 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:14.127891 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cd3a9-75f3-467a-926b-ca753f4c8141 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:14.128391 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cd3a9-75f3-467a-926b-ca753f4c8141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:14.169630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-b62cd3a9-75f3-467a-926b-ca753f4c8141 None None] MFA Rules not processed for user `a2013e1e0b674a90889c1f4462aeff75`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:14.171422 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b62cd3a9-75f3-467a-926b-ca753f4c8141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:14.234722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125faca-d08b-4026-8183-14cc7e2b34b7 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:14.236484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0125faca-d08b-4026-8183-14cc7e2b34b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:14.334113 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1047/2079] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:14.348171 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62cd3a9-75f3-467a-926b-ca753f4c8141 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:14.349695 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b62cd3a9-75f3-467a-926b-ca753f4c8141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:14.362077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125faca-d08b-4026-8183-14cc7e2b34b7 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:14.368123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684b560-dbfc-4bbb-9ad3-37c3197b0179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:14.369031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684b560-dbfc-4bbb-9ad3-37c3197b0179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:14.369512 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684b560-dbfc-4bbb-9ad3-37c3197b0179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:14.369973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684b560-dbfc-4bbb-9ad3-37c3197b0179 admin admin] PATH_INFO: `/v3/projects/5d1057cd915f42ed9744e648d7ff579a` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:14.371441 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684b560-dbfc-4bbb-9ad3-37c3197b0179 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d1057cd915f42ed9744e648d7ff579a)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:14.374292 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684b560-dbfc-4bbb-9ad3-37c3197b0179 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:14.390554 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:14.441416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62cd3a9-75f3-467a-926b-ca753f4c8141 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:14.445373 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:14.466800 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a684b560-dbfc-4bbb-9ad3-37c3197b0179 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d1057cd915f42ed9744e648d7ff579a', 'request_id': 'req-a684b560-dbfc-4bbb-9ad3-37c3197b0179'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:14.472192 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a684b560-dbfc-4bbb-9ad3-37c3197b0179 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d1057cd915f42ed9744e648d7ff579a', 'request_id': 'req-a684b560-dbfc-4bbb-9ad3-37c3197b0179'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:14.483414 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a684b560-dbfc-4bbb-9ad3-37c3197b0179 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d1057cd915f42ed9744e648d7ff579a', 'request_id': 'req-a684b560-dbfc-4bbb-9ad3-37c3197b0179'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:14.506562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a684b560-dbfc-4bbb-9ad3-37c3197b0179 admin admin] The token cache is being invalidate because project 5d1057cd915f42ed9744e648d7ff579a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:37:14.508377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a684b560-dbfc-4bbb-9ad3-37c3197b0179 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:14.509263 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a684b560-dbfc-4bbb-9ad3-37c3197b0179 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:14.520972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1033/2080] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:37:14 2022] DELETE /identity/v3/projects/5d1057cd915f42ed9744e648d7ff579a => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:14.521683 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684b560-dbfc-4bbb-9ad3-37c3197b0179 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:14.529301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-a684b560-dbfc-4bbb-9ad3-37c3197b0179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:14.563856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4ed23-ee74-4097-87ee-40839c644de2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:14.565140 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4ed23-ee74-4097-87ee-40839c644de2 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:14.565498 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4ed23-ee74-4097-87ee-40839c644de2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:14.566332 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4ed23-ee74-4097-87ee-40839c644de2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:14.568698 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-75e4ed23-ee74-4097-87ee-40839c644de2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:14.645030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684b560-dbfc-4bbb-9ad3-37c3197b0179 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:14.651574 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d6af5-814c-451e-97ee-f282bdca2970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:14.651769 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4ed23-ee74-4097-87ee-40839c644de2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:14.652242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d6af5-814c-451e-97ee-f282bdca2970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:14.652527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d6af5-814c-451e-97ee-f282bdca2970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:14.652805 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d6af5-814c-451e-97ee-f282bdca2970 admin admin] PATH_INFO: `/v3/projects/4f47d0ec4624485ab891405760ac8823` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:14.654267 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4ed23-ee74-4097-87ee-40839c644de2 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:14.654677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d6af5-814c-451e-97ee-f282bdca2970 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f47d0ec4624485ab891405760ac8823)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:14.656727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d6af5-814c-451e-97ee-f282bdca2970 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:14.667522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1048/2081] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:37:14 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:37:14.669791 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5914b-d7bd-4bc9-b3c8-997dd97ee102 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:14.670182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5914b-d7bd-4bc9-b3c8-997dd97ee102 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:14.670713 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5914b-d7bd-4bc9-b3c8-997dd97ee102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:14.698670 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-84d5914b-d7bd-4bc9-b3c8-997dd97ee102 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:14.704925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-84d5914b-d7bd-4bc9-b3c8-997dd97ee102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:14.714362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1049/2082] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:14.723123 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1bfd1-330d-4194-8c71-5ee2deff0059 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:14.723417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1bfd1-330d-4194-8c71-5ee2deff0059 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:14.723708 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1bfd1-330d-4194-8c71-5ee2deff0059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:14.745534 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-1dd1bfd1-330d-4194-8c71-5ee2deff0059 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:14.747374 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-1dd1bfd1-330d-4194-8c71-5ee2deff0059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:14.747565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-319d6af5-814c-451e-97ee-f282bdca2970 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f47d0ec4624485ab891405760ac8823', 'request_id': 'req-319d6af5-814c-451e-97ee-f282bdca2970'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:14.754778 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-319d6af5-814c-451e-97ee-f282bdca2970 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f47d0ec4624485ab891405760ac8823', 'request_id': 'req-319d6af5-814c-451e-97ee-f282bdca2970'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:14.763485 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-319d6af5-814c-451e-97ee-f282bdca2970 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f47d0ec4624485ab891405760ac8823', 'request_id': 'req-319d6af5-814c-451e-97ee-f282bdca2970'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:14.775523 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-319d6af5-814c-451e-97ee-f282bdca2970 admin admin] The token cache is being invalidate because project 4f47d0ec4624485ab891405760ac8823 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:37:14.775908 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-319d6af5-814c-451e-97ee-f282bdca2970 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:14.776555 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-319d6af5-814c-451e-97ee-f282bdca2970 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:14.778536 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1034/2083] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:37:14 2022] DELETE /identity/v3/projects/4f47d0ec4624485ab891405760ac8823 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:14.783062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f39f2-36c6-4490-9ad6-b0e9b1ab2360 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:14.783353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f39f2-36c6-4490-9ad6-b0e9b1ab2360 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:14.783638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f39f2-36c6-4490-9ad6-b0e9b1ab2360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:14.805031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-149f39f2-36c6-4490-9ad6-b0e9b1ab2360 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:14.806455 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-149f39f2-36c6-4490-9ad6-b0e9b1ab2360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:14.822347 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1050/2084] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:37:14.826996 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1bfd1-330d-4194-8c71-5ee2deff0059 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:14.836622 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1bfd1-330d-4194-8c71-5ee2deff0059 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:14.842167 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c8f89-e109-405d-8387-5cfeecce9f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:14.842856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c8f89-e109-405d-8387-5cfeecce9f2e admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:14.843210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c8f89-e109-405d-8387-5cfeecce9f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:14.843598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c8f89-e109-405d-8387-5cfeecce9f2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:14.844706 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c8f89-e109-405d-8387-5cfeecce9f2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:37:14.845260 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c8f89-e109-405d-8387-5cfeecce9f2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:14.846977 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c8f89-e109-405d-8387-5cfeecce9f2e admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:14.864074 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1051/2085] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:37:14.865148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:14.866812 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c8f89-e109-405d-8387-5cfeecce9f2e admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:14.886443 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c8f89-e109-405d-8387-5cfeecce9f2e admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:14.890845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfc3de-a2d1-4c28-8c42-db1fffd2d87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:14.891502 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfc3de-a2d1-4c28-8c42-db1fffd2d87a admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:14.891784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfc3de-a2d1-4c28-8c42-db1fffd2d87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:14.891917 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1035/2086] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:37:14.892122 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfc3de-a2d1-4c28-8c42-db1fffd2d87a admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:14.892872 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccfc3de-a2d1-4c28-8c42-db1fffd2d87a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:14.894814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccfc3de-a2d1-4c28-8c42-db1fffd2d87a admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:14.903021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e1fab-989b-4c59-aa2f-62f13ec6787f None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:14.904374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e1fab-989b-4c59-aa2f-62f13ec6787f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:14.904765 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e1fab-989b-4c59-aa2f-62f13ec6787f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:14.927642 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:14.927951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-be4e1fab-989b-4c59-aa2f-62f13ec6787f None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:14.929876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-be4e1fab-989b-4c59-aa2f-62f13ec6787f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:14.940206 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1052/2087] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:37:14 2022] POST /identity/v3/projects => generated 377 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:37:14.942434 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1036/2088] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:14.944527 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfc3de-a2d1-4c28-8c42-db1fffd2d87a admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:14.946848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e1fab-989b-4c59-aa2f-62f13ec6787f None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:14.962830 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e1fab-989b-4c59-aa2f-62f13ec6787f None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:15.032806 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfc3de-a2d1-4c28-8c42-db1fffd2d87a admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:15.041399 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9a508-e549-478a-88ae-a86a1ec0693d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:15.042491 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9a508-e549-478a-88ae-a86a1ec0693d admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:15.042824 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9a508-e549-478a-88ae-a86a1ec0693d admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:15.043200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9a508-e549-478a-88ae-a86a1ec0693d admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:15.044702 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9a508-e549-478a-88ae-a86a1ec0693d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:15.049069 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a8918-9e00-4a47-b550-f134709138db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:15.049301 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9a508-e549-478a-88ae-a86a1ec0693d admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:15.050215 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a8918-9e00-4a47-b550-f134709138db admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:15.050546 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a8918-9e00-4a47-b550-f134709138db admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:15.051050 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a8918-9e00-4a47-b550-f134709138db admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:15.051709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a8918-9e00-4a47-b550-f134709138db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:37:15.052063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a8918-9e00-4a47-b550-f134709138db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:15.057436 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a8918-9e00-4a47-b550-f134709138db admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:15.076246 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1037/2089] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:37:15.081317 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a8918-9e00-4a47-b550-f134709138db admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:15.083881 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1053/2090] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:37:14 2022] POST /identity/v3/users => generated 289 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:37:15.088234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9a508-e549-478a-88ae-a86a1ec0693d admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:15.101863 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9a508-e549-478a-88ae-a86a1ec0693d admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:15.102267 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a8918-9e00-4a47-b550-f134709138db admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:15.107633 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e4b0b-d35a-422e-a868-30d87c630a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:15.108065 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754bb707-c34b-4efa-b0d3-51d8557fbd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:15.108369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e4b0b-d35a-422e-a868-30d87c630a38 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:15.108653 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e4b0b-d35a-422e-a868-30d87c630a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:15.108949 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e4b0b-d35a-422e-a868-30d87c630a38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:15.109596 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754bb707-c34b-4efa-b0d3-51d8557fbd0b admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:15.109712 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e4b0b-d35a-422e-a868-30d87c630a38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:15.109992 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754bb707-c34b-4efa-b0d3-51d8557fbd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:15.110422 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754bb707-c34b-4efa-b0d3-51d8557fbd0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:15.111088 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754bb707-c34b-4efa-b0d3-51d8557fbd0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:15.111522 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e4b0b-d35a-422e-a868-30d87c630a38 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:15.112794 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754bb707-c34b-4efa-b0d3-51d8557fbd0b admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:15.124481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1054/2091] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:37:15.128192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754bb707-c34b-4efa-b0d3-51d8557fbd0b admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:15.138605 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754bb707-c34b-4efa-b0d3-51d8557fbd0b admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:15.143575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae0f5a-99d1-4140-9fff-511435f7b7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:15.144367 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae0f5a-99d1-4140-9fff-511435f7b7ee admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:15.144828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae0f5a-99d1-4140-9fff-511435f7b7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:15.145309 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae0f5a-99d1-4140-9fff-511435f7b7ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:15.146173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae0f5a-99d1-4140-9fff-511435f7b7ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:15.148626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae0f5a-99d1-4140-9fff-511435f7b7ee admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:15.150987 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1038/2092] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:37:15 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:37:15.151908 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e4b0b-d35a-422e-a868-30d87c630a38 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:15.166922 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1055/2093] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:37:15.166922 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae0f5a-99d1-4140-9fff-511435f7b7ee admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:15.166922 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-ddae0f5a-99d1-4140-9fff-511435f7b7ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:15.180743 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae0f5a-99d1-4140-9fff-511435f7b7ee admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:15.224107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e4b0b-d35a-422e-a868-30d87c630a38 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:15.229851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de0616-1764-4a7f-af30-87de8f21c55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:15.230595 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de0616-1764-4a7f-af30-87de8f21c55e admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:15.231061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de0616-1764-4a7f-af30-87de8f21c55e admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:15.231957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de0616-1764-4a7f-af30-87de8f21c55e admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:15.232720 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de0616-1764-4a7f-af30-87de8f21c55e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:15.234902 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de0616-1764-4a7f-af30-87de8f21c55e admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:15.238334 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb14b79-81fb-427e-a6f3-df22009bc3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:15.239017 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb14b79-81fb-427e-a6f3-df22009bc3a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:15.239324 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb14b79-81fb-427e-a6f3-df22009bc3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:15.239604 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb14b79-81fb-427e-a6f3-df22009bc3a6 admin admin] PATH_INFO: `/v3/users/90a59c8caadb4242b7fca6c146a8cd20` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:15.240928 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb14b79-81fb-427e-a6f3-df22009bc3a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90a59c8caadb4242b7fca6c146a8cd20)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:15.242823 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb14b79-81fb-427e-a6f3-df22009bc3a6 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:15.256706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1039/2094] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:37:15 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:37:15.257530 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de0616-1764-4a7f-af30-87de8f21c55e admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:15.267859 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de0616-1764-4a7f-af30-87de8f21c55e admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:15.273426 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1dc511-b354-4c40-a06c-e8924b1201fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:15.274124 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1dc511-b354-4c40-a06c-e8924b1201fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:15.274407 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1dc511-b354-4c40-a06c-e8924b1201fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:15.274685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1dc511-b354-4c40-a06c-e8924b1201fb admin admin] PATH_INFO: `/v3/projects/bd543689d3ca415fb6cbb077489d07f3/users/287ffcd2b7df439c9e5a4dc65535eb0d/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:15.292600 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1dc511-b354-4c40-a06c-e8924b1201fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd543689d3ca415fb6cbb077489d07f3, user_id=287ffcd2b7df439c9e5a4dc65535eb0d, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:15.296052 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1dc511-b354-4c40-a06c-e8924b1201fb admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:15.300245 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-cfb14b79-81fb-427e-a6f3-df22009bc3a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90a59c8caadb4242b7fca6c146a8cd20'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:15.309925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-cfb14b79-81fb-427e-a6f3-df22009bc3a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90a59c8caadb4242b7fca6c146a8cd20'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:15.313733 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1040/2095] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:37:15 2022] PUT /identity/v3/projects/bd543689d3ca415fb6cbb077489d07f3/users/287ffcd2b7df439c9e5a4dc65535eb0d/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:15.323206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1dc511-b354-4c40-a06c-e8924b1201fb admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:15.324238 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:15.327075 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-cfb14b79-81fb-427e-a6f3-df22009bc3a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90a59c8caadb4242b7fca6c146a8cd20'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:15.335990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-cfb14b79-81fb-427e-a6f3-df22009bc3a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90a59c8caadb4242b7fca6c146a8cd20'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:15.337423 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-cfb14b79-81fb-427e-a6f3-df22009bc3a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90a59c8caadb4242b7fca6c146a8cd20'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:15.348946 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1056/2096] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:37:15 2022] DELETE /identity/v3/users/90a59c8caadb4242b7fca6c146a8cd20 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:15.352871 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21b9f8-7e24-4f5f-b1aa-8636f4646b8d None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:15.353253 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21b9f8-7e24-4f5f-b1aa-8636f4646b8d None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:15.353542 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21b9f8-7e24-4f5f-b1aa-8636f4646b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:15.354156 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:15.377892 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-2b21b9f8-7e24-4f5f-b1aa-8636f4646b8d None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:15.379139 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-2b21b9f8-7e24-4f5f-b1aa-8636f4646b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:15.402584 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1dc511-b354-4c40-a06c-e8924b1201fb admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:15.442442 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1057/2097] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:37:15.443650 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67748b23-c77d-4cc9-af79-ffaad1eb42e1 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:15.443943 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67748b23-c77d-4cc9-af79-ffaad1eb42e1 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:15.444225 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67748b23-c77d-4cc9-af79-ffaad1eb42e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:15.462772 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a728a9-ac4a-4e0e-ac5b-c874ed0ea5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:15.463518 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a728a9-ac4a-4e0e-ac5b-c874ed0ea5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:15.464078 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a728a9-ac4a-4e0e-ac5b-c874ed0ea5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:15.464357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a728a9-ac4a-4e0e-ac5b-c874ed0ea5a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:15.465097 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a728a9-ac4a-4e0e-ac5b-c874ed0ea5a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:15.467220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a728a9-ac4a-4e0e-ac5b-c874ed0ea5a2 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:15.474346 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-67748b23-c77d-4cc9-af79-ffaad1eb42e1 None None] MFA Rules not processed for user `287ffcd2b7df439c9e5a4dc65535eb0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:15.475555 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-67748b23-c77d-4cc9-af79-ffaad1eb42e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:15.480536 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1041/2098] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:37:15.480536 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a728a9-ac4a-4e0e-ac5b-c874ed0ea5a2 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:15.482876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-43a728a9-ac4a-4e0e-ac5b-c874ed0ea5a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:15.539671 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a728a9-ac4a-4e0e-ac5b-c874ed0ea5a2 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:15.542006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1058/2099] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:15.546467 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67748b23-c77d-4cc9-af79-ffaad1eb42e1 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:15.546467 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-67748b23-c77d-4cc9-af79-ffaad1eb42e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:15.548433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eddec3-c93b-4578-a3da-2834f78fe24d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:15.548660 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eddec3-c93b-4578-a3da-2834f78fe24d service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:15.548971 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eddec3-c93b-4578-a3da-2834f78fe24d service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:15.549258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eddec3-c93b-4578-a3da-2834f78fe24d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:15.558786 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eddec3-c93b-4578-a3da-2834f78fe24d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:15.560264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eddec3-c93b-4578-a3da-2834f78fe24d service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:15.560921 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67748b23-c77d-4cc9-af79-ffaad1eb42e1 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:15.566846 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279dd815-17d6-4473-86fc-b924091edcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:15.567458 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dd815-17d6-4473-86fc-b924091edcc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:15.567737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dd815-17d6-4473-86fc-b924091edcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:15.568028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dd815-17d6-4473-86fc-b924091edcc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:15.568715 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279dd815-17d6-4473-86fc-b924091edcc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:15.570201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1042/2100] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:37:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:37:15.570347 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279dd815-17d6-4473-86fc-b924091edcc8 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:15.573759 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eddec3-c93b-4578-a3da-2834f78fe24d service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:15.575278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-e5eddec3-c93b-4578-a3da-2834f78fe24d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:15.583794 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1059/2101] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:37:15.584790 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279dd815-17d6-4473-86fc-b924091edcc8 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:15.596099 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279dd815-17d6-4473-86fc-b924091edcc8 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:15.601655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833a863-9aa6-4d46-bb95-7190b0296c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:15.602796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833a863-9aa6-4d46-bb95-7190b0296c06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:15.603075 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833a863-9aa6-4d46-bb95-7190b0296c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:15.603352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833a863-9aa6-4d46-bb95-7190b0296c06 admin admin] PATH_INFO: `/v3/projects/4c98d056a74c4debb76497a087e6bb0b/users/6e10765d910341ad92a4fd640d405603/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:15.621342 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f833a863-9aa6-4d46-bb95-7190b0296c06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c98d056a74c4debb76497a087e6bb0b, user_id=6e10765d910341ad92a4fd640d405603, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:15.624506 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f833a863-9aa6-4d46-bb95-7190b0296c06 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:15.641791 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1060/2102] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:37:15 2022] PUT /identity/v3/projects/4c98d056a74c4debb76497a087e6bb0b/users/6e10765d910341ad92a4fd640d405603/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:37:15.652076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd4494-0cc3-4251-ad5e-c17ac75109a9 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:15.652399 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd4494-0cc3-4251-ad5e-c17ac75109a9 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:15.652701 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd4494-0cc3-4251-ad5e-c17ac75109a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:15.655766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eddec3-c93b-4578-a3da-2834f78fe24d service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:15.676970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:15.683758 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-65bd4494-0cc3-4251-ad5e-c17ac75109a9 None None] MFA Rules not processed for user `6e10765d910341ad92a4fd640d405603`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:15.685154 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-65bd4494-0cc3-4251-ad5e-c17ac75109a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:15.719661 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a370520-d06e-4db6-83e4-75c05b34ff52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:15.720687 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a370520-d06e-4db6-83e4-75c05b34ff52 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:15.721152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a370520-d06e-4db6-83e4-75c05b34ff52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:15.721523 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a370520-d06e-4db6-83e4-75c05b34ff52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:15.723876 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:15.778317 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1061/2103] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:37:15.779923 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee0173-e866-4c39-83a3-311b9421eee9 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:15.780439 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee0173-e866-4c39-83a3-311b9421eee9 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:15.781021 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee0173-e866-4c39-83a3-311b9421eee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:15.819015 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-09ee0173-e866-4c39-83a3-311b9421eee9 None None] MFA Rules not processed for user `4fe9ea35543546afab739c0cfa3389fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:15.820603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-09ee0173-e866-4c39-83a3-311b9421eee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:15.821254 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a370520-d06e-4db6-83e4-75c05b34ff52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:15.823289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a370520-d06e-4db6-83e4-75c05b34ff52 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:15.848188 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1043/2104] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:37:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:37:15.849583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a370520-d06e-4db6-83e4-75c05b34ff52 service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:15.850927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-1a370520-d06e-4db6-83e4-75c05b34ff52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:15.876405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a370520-d06e-4db6-83e4-75c05b34ff52 service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:15.888671 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b6b18-16c7-4004-8184-bfdf0c874f00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:15.890894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b6b18-16c7-4004-8184-bfdf0c874f00 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:15.891697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b6b18-16c7-4004-8184-bfdf0c874f00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:15.893017 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b6b18-16c7-4004-8184-bfdf0c874f00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:15.911510 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b6b18-16c7-4004-8184-bfdf0c874f00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:15.913908 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b6b18-16c7-4004-8184-bfdf0c874f00 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:15.928281 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1044/2105] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:37:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:37:15.953353 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1062/2106] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:15.975877 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b6b18-16c7-4004-8184-bfdf0c874f00 service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:15.977475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-e74b6b18-16c7-4004-8184-bfdf0c874f00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:16.085224 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b6b18-16c7-4004-8184-bfdf0c874f00 service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:16.091960 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc04569-4015-482d-a7e2-098b0418994c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:16.092615 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc04569-4015-482d-a7e2-098b0418994c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:16.092920 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc04569-4015-482d-a7e2-098b0418994c admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:16.093213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc04569-4015-482d-a7e2-098b0418994c admin admin] PATH_INFO: `/v3/projects/cd9632a21e414bb18333374c294af0eb` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:16.094627 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc04569-4015-482d-a7e2-098b0418994c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd9632a21e414bb18333374c294af0eb)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:16.096712 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc04569-4015-482d-a7e2-098b0418994c admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:16.167517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-efc04569-4015-482d-a7e2-098b0418994c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd9632a21e414bb18333374c294af0eb', 'request_id': 'req-efc04569-4015-482d-a7e2-098b0418994c'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:16.177852 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-efc04569-4015-482d-a7e2-098b0418994c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd9632a21e414bb18333374c294af0eb', 'request_id': 'req-efc04569-4015-482d-a7e2-098b0418994c'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:16.186857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-efc04569-4015-482d-a7e2-098b0418994c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd9632a21e414bb18333374c294af0eb', 'request_id': 'req-efc04569-4015-482d-a7e2-098b0418994c'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:16.188429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:16.204267 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-efc04569-4015-482d-a7e2-098b0418994c admin admin] The token cache is being invalidate because project cd9632a21e414bb18333374c294af0eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:37:16.204830 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-efc04569-4015-482d-a7e2-098b0418994c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:16.205840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-efc04569-4015-482d-a7e2-098b0418994c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:16.208881 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1045/2107] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:37:15 2022] DELETE /identity/v3/projects/cd9632a21e414bb18333374c294af0eb => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:16.216155 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee0173-e866-4c39-83a3-311b9421eee9 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:16.228854 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee0173-e866-4c39-83a3-311b9421eee9 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:16.318875 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01936e7-9ec0-493e-a7fa-e4b0696ed4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:16.319532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01936e7-9ec0-493e-a7fa-e4b0696ed4ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:16.319811 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01936e7-9ec0-493e-a7fa-e4b0696ed4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:16.320085 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01936e7-9ec0-493e-a7fa-e4b0696ed4ca admin admin] PATH_INFO: `/v3/users/a2013e1e0b674a90889c1f4462aeff75` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:16.323546 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01936e7-9ec0-493e-a7fa-e4b0696ed4ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2013e1e0b674a90889c1f4462aeff75)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:16.325707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01936e7-9ec0-493e-a7fa-e4b0696ed4ca admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:16.393016 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a01936e7-9ec0-493e-a7fa-e4b0696ed4ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2013e1e0b674a90889c1f4462aeff75'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:16.405370 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a01936e7-9ec0-493e-a7fa-e4b0696ed4ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2013e1e0b674a90889c1f4462aeff75'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:16.438524 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a01936e7-9ec0-493e-a7fa-e4b0696ed4ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2013e1e0b674a90889c1f4462aeff75'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:16.444076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:16.450684 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a01936e7-9ec0-493e-a7fa-e4b0696ed4ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2013e1e0b674a90889c1f4462aeff75'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:16.451878 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a01936e7-9ec0-493e-a7fa-e4b0696ed4ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2013e1e0b674a90889c1f4462aeff75'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:16.465065 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1063/2108] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:37:16 2022] DELETE /identity/v3/users/a2013e1e0b674a90889c1f4462aeff75 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:16.817342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc04569-4015-482d-a7e2-098b0418994c admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:16.819009 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-efc04569-4015-482d-a7e2-098b0418994c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:16.836081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc04569-4015-482d-a7e2-098b0418994c admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:16.895315 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576cd6b-387f-4b41-b3a8-df11409ef375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:16.895980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576cd6b-387f-4b41-b3a8-df11409ef375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:16.896221 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576cd6b-387f-4b41-b3a8-df11409ef375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:16.897162 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576cd6b-387f-4b41-b3a8-df11409ef375 admin admin] PATH_INFO: `/v3/projects/0540193d30f14c418991f3c2e0465670` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:16.898367 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576cd6b-387f-4b41-b3a8-df11409ef375 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0540193d30f14c418991f3c2e0465670)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:16.900426 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576cd6b-387f-4b41-b3a8-df11409ef375 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:16.950869 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-5576cd6b-387f-4b41-b3a8-df11409ef375 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0540193d30f14c418991f3c2e0465670', 'request_id': 'req-5576cd6b-387f-4b41-b3a8-df11409ef375'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:16.957725 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-5576cd6b-387f-4b41-b3a8-df11409ef375 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0540193d30f14c418991f3c2e0465670', 'request_id': 'req-5576cd6b-387f-4b41-b3a8-df11409ef375'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:16.972995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-5576cd6b-387f-4b41-b3a8-df11409ef375 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0540193d30f14c418991f3c2e0465670', 'request_id': 'req-5576cd6b-387f-4b41-b3a8-df11409ef375'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:16.989816 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-5576cd6b-387f-4b41-b3a8-df11409ef375 admin admin] The token cache is being invalidate because project 0540193d30f14c418991f3c2e0465670 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:37:16.990046 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-5576cd6b-387f-4b41-b3a8-df11409ef375 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:16.990693 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-5576cd6b-387f-4b41-b3a8-df11409ef375 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:16.992469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1046/2109] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:37:16 2022] DELETE /identity/v3/projects/0540193d30f14c418991f3c2e0465670 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:16.998551 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea060204-afd1-458a-8dab-f10eda0a75fd None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:16.999052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea060204-afd1-458a-8dab-f10eda0a75fd None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:16.999368 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea060204-afd1-458a-8dab-f10eda0a75fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:17.022533 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-ea060204-afd1-458a-8dab-f10eda0a75fd None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:17.023763 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-ea060204-afd1-458a-8dab-f10eda0a75fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:17.095267 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1064/2110] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:17.104327 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca799f-9af6-4124-8d48-891c5f7daa39 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:17.104792 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca799f-9af6-4124-8d48-891c5f7daa39 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:17.105176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca799f-9af6-4124-8d48-891c5f7daa39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:17.132020 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-abca799f-9af6-4124-8d48-891c5f7daa39 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:17.133472 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-abca799f-9af6-4124-8d48-891c5f7daa39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:17.143216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1047/2111] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:17.148565 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea060204-afd1-458a-8dab-f10eda0a75fd None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:17.159436 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea060204-afd1-458a-8dab-f10eda0a75fd None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:17.171993 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0929720-f9a3-4bd4-8ca8-2accd536d27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:17.172801 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0929720-f9a3-4bd4-8ca8-2accd536d27b admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:17.172958 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0929720-f9a3-4bd4-8ca8-2accd536d27b admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:17.173333 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0929720-f9a3-4bd4-8ca8-2accd536d27b admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:17.174073 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0929720-f9a3-4bd4-8ca8-2accd536d27b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:37:17.174376 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0929720-f9a3-4bd4-8ca8-2accd536d27b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:17.176182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0929720-f9a3-4bd4-8ca8-2accd536d27b admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:17.191127 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1065/2112] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:37:17.195269 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca799f-9af6-4124-8d48-891c5f7daa39 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:17.205353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca799f-9af6-4124-8d48-891c5f7daa39 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:17.210082 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec63e5-dc7b-4870-893e-c009b67b4edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:17.210702 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec63e5-dc7b-4870-893e-c009b67b4edc admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:17.210986 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec63e5-dc7b-4870-893e-c009b67b4edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:17.211261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec63e5-dc7b-4870-893e-c009b67b4edc admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:17.212189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec63e5-dc7b-4870-893e-c009b67b4edc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:17.214622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec63e5-dc7b-4870-893e-c009b67b4edc admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:17.244421 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1048/2113] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:37:17 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:37:17.249077 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0929720-f9a3-4bd4-8ca8-2accd536d27b admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:17.259133 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0929720-f9a3-4bd4-8ca8-2accd536d27b admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:17.325364 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8299f-d779-4c60-b945-2b03f3a41422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:17.326322 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8299f-d779-4c60-b945-2b03f3a41422 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:17.326613 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8299f-d779-4c60-b945-2b03f3a41422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:17.340244 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8299f-d779-4c60-b945-2b03f3a41422 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:17.340244 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a8299f-d779-4c60-b945-2b03f3a41422 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:17.340244 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a8299f-d779-4c60-b945-2b03f3a41422 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:17.360417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1066/2114] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:37:17 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:37:17.361806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec63e5-dc7b-4870-893e-c009b67b4edc admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:17.373153 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec63e5-dc7b-4870-893e-c009b67b4edc admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:17.377835 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce69d84-0a1f-469e-8390-e49b0dbe9eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:17.378447 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce69d84-0a1f-469e-8390-e49b0dbe9eca admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:17.378735 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce69d84-0a1f-469e-8390-e49b0dbe9eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:17.379011 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce69d84-0a1f-469e-8390-e49b0dbe9eca admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:17.379715 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce69d84-0a1f-469e-8390-e49b0dbe9eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:17.381345 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce69d84-0a1f-469e-8390-e49b0dbe9eca admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:17.392925 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1049/2115] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:37:17.397002 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8299f-d779-4c60-b945-2b03f3a41422 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:17.409353 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8299f-d779-4c60-b945-2b03f3a41422 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:17.414614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc023bea-9757-4391-9f25-5658aa9689b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:17.415233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc023bea-9757-4391-9f25-5658aa9689b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:17.415537 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc023bea-9757-4391-9f25-5658aa9689b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:17.415808 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc023bea-9757-4391-9f25-5658aa9689b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:17.416504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc023bea-9757-4391-9f25-5658aa9689b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:17.418240 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc023bea-9757-4391-9f25-5658aa9689b6 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:17.431972 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1067/2116] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:37:17.436603 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce69d84-0a1f-469e-8390-e49b0dbe9eca admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:17.452820 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce69d84-0a1f-469e-8390-e49b0dbe9eca admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:17.457600 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c026e0b-97be-4092-912d-be9e215de042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:17.458265 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c026e0b-97be-4092-912d-be9e215de042 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:17.458542 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c026e0b-97be-4092-912d-be9e215de042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:17.458819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c026e0b-97be-4092-912d-be9e215de042 admin admin] PATH_INFO: `/v3/projects/0de76c9908a24002a9a5ae43249489f8/users/13b5c9d74a4f4fc99f0be66e47e53c7a/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:17.479742 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c026e0b-97be-4092-912d-be9e215de042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de76c9908a24002a9a5ae43249489f8, user_id=13b5c9d74a4f4fc99f0be66e47e53c7a, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:17.484088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c026e0b-97be-4092-912d-be9e215de042 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:17.519141 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1050/2117] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:37:17 2022] PUT /identity/v3/projects/0de76c9908a24002a9a5ae43249489f8/users/13b5c9d74a4f4fc99f0be66e47e53c7a/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:17.530626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d01e01-a79f-4f11-a3f0-00514553a690 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:17.530899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d01e01-a79f-4f11-a3f0-00514553a690 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:17.531672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d01e01-a79f-4f11-a3f0-00514553a690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:17.565264 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-e9d01e01-a79f-4f11-a3f0-00514553a690 None None] MFA Rules not processed for user `13b5c9d74a4f4fc99f0be66e47e53c7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:17.566489 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e9d01e01-a79f-4f11-a3f0-00514553a690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:17.617925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:17.643115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1068/2118] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:17.644560 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c026e0b-97be-4092-912d-be9e215de042 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:17.678927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:17.732059 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c026e0b-97be-4092-912d-be9e215de042 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:17.737481 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41ad16-2b6d-46a7-9326-eb557faf40c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:17.738181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41ad16-2b6d-46a7-9326-eb557faf40c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:17.738538 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41ad16-2b6d-46a7-9326-eb557faf40c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:17.738888 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41ad16-2b6d-46a7-9326-eb557faf40c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:17.739636 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41ad16-2b6d-46a7-9326-eb557faf40c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:17.741617 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41ad16-2b6d-46a7-9326-eb557faf40c5 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:17.774355 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1051/2119] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:37:17 2022] POST /identity/v3/projects => generated 375 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:37:17.779028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d01e01-a79f-4f11-a3f0-00514553a690 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:17.790197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d01e01-a79f-4f11-a3f0-00514553a690 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:17.863844 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44ff04-5b37-40d5-b1cb-8d72f4d3bf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:17.864689 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44ff04-5b37-40d5-b1cb-8d72f4d3bf65 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:17.865181 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44ff04-5b37-40d5-b1cb-8d72f4d3bf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:17.865623 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44ff04-5b37-40d5-b1cb-8d72f4d3bf65 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:17.866451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44ff04-5b37-40d5-b1cb-8d72f4d3bf65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:17.868707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44ff04-5b37-40d5-b1cb-8d72f4d3bf65 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:17.901055 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1069/2120] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:37:17 2022] POST /identity/v3/users => generated 294 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:37:17.906668 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41ad16-2b6d-46a7-9326-eb557faf40c5 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:17.917432 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41ad16-2b6d-46a7-9326-eb557faf40c5 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:17.922973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bc8fc-ab54-4535-9007-409bfc986ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:17.923697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bc8fc-ab54-4535-9007-409bfc986ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:17.923976 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bc8fc-ab54-4535-9007-409bfc986ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:17.924249 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bc8fc-ab54-4535-9007-409bfc986ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:17.924972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2bc8fc-ab54-4535-9007-409bfc986ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:17.926567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2bc8fc-ab54-4535-9007-409bfc986ca0 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:17.938027 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1052/2121] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:37:17.942482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44ff04-5b37-40d5-b1cb-8d72f4d3bf65 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:17.953479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44ff04-5b37-40d5-b1cb-8d72f4d3bf65 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:17.958073 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0969a74-fab7-4d41-87c0-666de94959a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:17.958728 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0969a74-fab7-4d41-87c0-666de94959a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:17.959007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0969a74-fab7-4d41-87c0-666de94959a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:17.959280 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0969a74-fab7-4d41-87c0-666de94959a4 admin admin] PATH_INFO: `/v3/projects/0928b23755fc4186b04c65dade0f7f20/users/6c1556baf8974cbfaab5cfcefbb1acd9/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:17.974996 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0969a74-fab7-4d41-87c0-666de94959a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0928b23755fc4186b04c65dade0f7f20, user_id=6c1556baf8974cbfaab5cfcefbb1acd9, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:17.978231 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0969a74-fab7-4d41-87c0-666de94959a4 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:17.991367 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1070/2122] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:37:17 2022] PUT /identity/v3/projects/0928b23755fc4186b04c65dade0f7f20/users/6c1556baf8974cbfaab5cfcefbb1acd9/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:37:17.995295 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bc8fc-ab54-4535-9007-409bfc986ca0 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:18.005712 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bc8fc-ab54-4535-9007-409bfc986ca0 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:18.060709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9003e49-ce35-4049-bba4-d0d639a84c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:18.061355 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9003e49-ce35-4049-bba4-d0d639a84c96 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:18.061634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9003e49-ce35-4049-bba4-d0d639a84c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:18.061910 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9003e49-ce35-4049-bba4-d0d639a84c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:18.062588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9003e49-ce35-4049-bba4-d0d639a84c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:18.064185 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9003e49-ce35-4049-bba4-d0d639a84c96 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:18.073709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1053/2123] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:37:18.078456 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0969a74-fab7-4d41-87c0-666de94959a4 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:18.089183 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0969a74-fab7-4d41-87c0-666de94959a4 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:18.094592 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50bfd4-fd11-410c-a559-cd8ee025e2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:18.095253 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50bfd4-fd11-410c-a559-cd8ee025e2b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:18.095538 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50bfd4-fd11-410c-a559-cd8ee025e2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:18.095815 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50bfd4-fd11-410c-a559-cd8ee025e2b4 admin admin] PATH_INFO: `/v3/projects/0928b23755fc4186b04c65dade0f7f20/users/6c1556baf8974cbfaab5cfcefbb1acd9/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:18.097809 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50bfd4-fd11-410c-a559-cd8ee025e2b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0928b23755fc4186b04c65dade0f7f20, user_id=6c1556baf8974cbfaab5cfcefbb1acd9, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:18.100432 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50bfd4-fd11-410c-a559-cd8ee025e2b4 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:18.110470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1071/2124] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:37:18 2022] PUT /identity/v3/projects/0928b23755fc4186b04c65dade0f7f20/users/6c1556baf8974cbfaab5cfcefbb1acd9/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:18.121434 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6f91a-3532-407a-b73a-93c5b4460bc5 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:18.121741 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6f91a-3532-407a-b73a-93c5b4460bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:18.122094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6f91a-3532-407a-b73a-93c5b4460bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:18.153738 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-08c6f91a-3532-407a-b73a-93c5b4460bc5 None None] MFA Rules not processed for user `6c1556baf8974cbfaab5cfcefbb1acd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:18.153738 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-08c6f91a-3532-407a-b73a-93c5b4460bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:18.221927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1054/2125] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:37:18.233048 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50bfd4-fd11-410c-a559-cd8ee025e2b4 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:18.234846 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-cf50bfd4-fd11-410c-a559-cd8ee025e2b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:18.293376 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50bfd4-fd11-410c-a559-cd8ee025e2b4 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:18.297124 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dcf5e9-8c3a-4d0f-9406-520b3364dabc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:18.297744 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcf5e9-8c3a-4d0f-9406-520b3364dabc service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:18.298027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcf5e9-8c3a-4d0f-9406-520b3364dabc service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:18.298300 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcf5e9-8c3a-4d0f-9406-520b3364dabc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:18.351882 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:18.354247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dcf5e9-8c3a-4d0f-9406-520b3364dabc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:18.356534 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dcf5e9-8c3a-4d0f-9406-520b3364dabc service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:18.365827 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1072/2126] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:37:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:37:28.217189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=92,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:28.361890 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:31.412748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6f91a-3532-407a-b73a-93c5b4460bc5 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:31.423229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6f91a-3532-407a-b73a-93c5b4460bc5 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:31.430310 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1735a2e-5fdb-4bf7-b31d-7e524720dc78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:31.431095 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1735a2e-5fdb-4bf7-b31d-7e524720dc78 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:31.431482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1735a2e-5fdb-4bf7-b31d-7e524720dc78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:31.431863 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1735a2e-5fdb-4bf7-b31d-7e524720dc78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:31.444699 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1735a2e-5fdb-4bf7-b31d-7e524720dc78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:31.446412 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1735a2e-5fdb-4bf7-b31d-7e524720dc78 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:31.456832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1055/2127] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:37:31 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:37:31.872553 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dcf5e9-8c3a-4d0f-9406-520b3364dabc service cinder] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:31.874266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-96dcf5e9-8c3a-4d0f-9406-520b3364dabc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:31.956693 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dcf5e9-8c3a-4d0f-9406-520b3364dabc service cinder] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:31.962096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd04cdc-395c-4178-b495-b34237adde45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:31.962864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd04cdc-395c-4178-b495-b34237adde45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:31.963197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd04cdc-395c-4178-b495-b34237adde45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:31.963507 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd04cdc-395c-4178-b495-b34237adde45 admin admin] PATH_INFO: `/v3/users/6e10765d910341ad92a4fd640d405603` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:31.965018 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd04cdc-395c-4178-b495-b34237adde45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e10765d910341ad92a4fd640d405603)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:31.967469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd04cdc-395c-4178-b495-b34237adde45 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:32.048686 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-8bd04cdc-395c-4178-b495-b34237adde45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e10765d910341ad92a4fd640d405603'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:32.062068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-8bd04cdc-395c-4178-b495-b34237adde45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e10765d910341ad92a4fd640d405603'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:32.075660 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-8bd04cdc-395c-4178-b495-b34237adde45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e10765d910341ad92a4fd640d405603'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:32.087189 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-8bd04cdc-395c-4178-b495-b34237adde45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e10765d910341ad92a4fd640d405603'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:32.087948 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-8bd04cdc-395c-4178-b495-b34237adde45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e10765d910341ad92a4fd640d405603'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:32.094780 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1073/2128] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:37:31 2022] DELETE /identity/v3/users/6e10765d910341ad92a4fd640d405603 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:37:32.099527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5e989-d9e4-4c84-9740-8f1376873f5a None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:32.099817 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5e989-d9e4-4c84-9740-8f1376873f5a None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:32.100101 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5e989-d9e4-4c84-9740-8f1376873f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:32.119157 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-5fc5e989-d9e4-4c84-9740-8f1376873f5a None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:32.120391 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-5fc5e989-d9e4-4c84-9740-8f1376873f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:32.181294 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1056/2129] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:32.191677 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd04cdc-395c-4178-b495-b34237adde45 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:32.193188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-8bd04cdc-395c-4178-b495-b34237adde45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:32.250867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd04cdc-395c-4178-b495-b34237adde45 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:32.270878 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1b2fe-b183-40cb-ba18-9b0d7b38939d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:32.271533 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1b2fe-b183-40cb-ba18-9b0d7b38939d service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:32.271818 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1b2fe-b183-40cb-ba18-9b0d7b38939d service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:32.272098 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1b2fe-b183-40cb-ba18-9b0d7b38939d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:32.285145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1b2fe-b183-40cb-ba18-9b0d7b38939d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:32.286611 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1b2fe-b183-40cb-ba18-9b0d7b38939d service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:32.297658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1074/2130] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:37:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:37:32.668296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5e989-d9e4-4c84-9740-8f1376873f5a None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:32.671079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-5fc5e989-d9e4-4c84-9740-8f1376873f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:32.683760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5e989-d9e4-4c84-9740-8f1376873f5a None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:32.688687 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce595d6-7120-4343-ab24-a383fd760089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:32.689375 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce595d6-7120-4343-ab24-a383fd760089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:32.689661 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce595d6-7120-4343-ab24-a383fd760089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:32.689939 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce595d6-7120-4343-ab24-a383fd760089 admin admin] PATH_INFO: `/v3/projects/4c98d056a74c4debb76497a087e6bb0b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:32.691282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce595d6-7120-4343-ab24-a383fd760089 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c98d056a74c4debb76497a087e6bb0b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:32.693520 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce595d6-7120-4343-ab24-a383fd760089 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:32.742118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0ce595d6-7120-4343-ab24-a383fd760089 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c98d056a74c4debb76497a087e6bb0b', 'request_id': 'req-0ce595d6-7120-4343-ab24-a383fd760089'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:32.747523 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0ce595d6-7120-4343-ab24-a383fd760089 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c98d056a74c4debb76497a087e6bb0b', 'request_id': 'req-0ce595d6-7120-4343-ab24-a383fd760089'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:32.755241 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0ce595d6-7120-4343-ab24-a383fd760089 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c98d056a74c4debb76497a087e6bb0b', 'request_id': 'req-0ce595d6-7120-4343-ab24-a383fd760089'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:32.767777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0ce595d6-7120-4343-ab24-a383fd760089 admin admin] The token cache is being invalidate because project 4c98d056a74c4debb76497a087e6bb0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:37:32.768152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0ce595d6-7120-4343-ab24-a383fd760089 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:32.768797 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0ce595d6-7120-4343-ab24-a383fd760089 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:32.770407 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1057/2131] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:37:32 2022] DELETE /identity/v3/projects/4c98d056a74c4debb76497a087e6bb0b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:32.775515 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a850874-2e8b-424f-8218-41ef17a64abd None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:32.775829 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a850874-2e8b-424f-8218-41ef17a64abd None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:32.776102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a850874-2e8b-424f-8218-41ef17a64abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:32.797154 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-7a850874-2e8b-424f-8218-41ef17a64abd None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:32.798348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7a850874-2e8b-424f-8218-41ef17a64abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:32.817054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:32.870238 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1075/2132] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:32.878311 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc046a-0a85-4f99-8049-c8796e2e3cfb None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:32.878578 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc046a-0a85-4f99-8049-c8796e2e3cfb None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:32.878844 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc046a-0a85-4f99-8049-c8796e2e3cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:32.899581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-41bc046a-0a85-4f99-8049-c8796e2e3cfb None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:32.900779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-41bc046a-0a85-4f99-8049-c8796e2e3cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:32.909422 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1058/2133] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:32.913546 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a850874-2e8b-424f-8218-41ef17a64abd None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:32.923467 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a850874-2e8b-424f-8218-41ef17a64abd None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:32.928798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0124864-a4c7-4730-924d-123c6d8aac38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:32.929640 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0124864-a4c7-4730-924d-123c6d8aac38 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:32.929988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0124864-a4c7-4730-924d-123c6d8aac38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:32.930290 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0124864-a4c7-4730-924d-123c6d8aac38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:32.931185 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0124864-a4c7-4730-924d-123c6d8aac38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:37:32.931548 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0124864-a4c7-4730-924d-123c6d8aac38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:32.933325 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0124864-a4c7-4730-924d-123c6d8aac38 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:32.948540 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1076/2134] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:37:32.951412 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc046a-0a85-4f99-8049-c8796e2e3cfb None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:32.962282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc046a-0a85-4f99-8049-c8796e2e3cfb None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:32.975956 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8467b0a-b0ea-40de-923f-7e54cfc85747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:32.976763 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8467b0a-b0ea-40de-923f-7e54cfc85747 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:32.977158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8467b0a-b0ea-40de-923f-7e54cfc85747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:32.977781 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8467b0a-b0ea-40de-923f-7e54cfc85747 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:32.978318 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8467b0a-b0ea-40de-923f-7e54cfc85747 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:32.980645 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8467b0a-b0ea-40de-923f-7e54cfc85747 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:33.016050 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1059/2135] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:37:32 2022] POST /identity/v3/projects => generated 369 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:37:33.017709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0124864-a4c7-4730-924d-123c6d8aac38 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:33.028020 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0124864-a4c7-4730-924d-123c6d8aac38 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:33.084126 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4ab07-0ca7-4355-bddb-5eca40e6e2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:33.084661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4ab07-0ca7-4355-bddb-5eca40e6e2b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:33.084966 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4ab07-0ca7-4355-bddb-5eca40e6e2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:33.085236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4ab07-0ca7-4355-bddb-5eca40e6e2b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:33.085878 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4ab07-0ca7-4355-bddb-5eca40e6e2b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:33.087780 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4ab07-0ca7-4355-bddb-5eca40e6e2b4 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:33.113520 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1077/2136] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:37:33 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:37:33.114754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8467b0a-b0ea-40de-923f-7e54cfc85747 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:33.125615 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8467b0a-b0ea-40de-923f-7e54cfc85747 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:33.130516 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fd9e1-6e38-45d9-9552-4af69056c91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:33.131132 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fd9e1-6e38-45d9-9552-4af69056c91e admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:33.131413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fd9e1-6e38-45d9-9552-4af69056c91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:33.131686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fd9e1-6e38-45d9-9552-4af69056c91e admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:33.132372 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763fd9e1-6e38-45d9-9552-4af69056c91e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:33.133990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763fd9e1-6e38-45d9-9552-4af69056c91e admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:33.144592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1060/2137] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:37:33.148574 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4ab07-0ca7-4355-bddb-5eca40e6e2b4 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:33.159908 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4ab07-0ca7-4355-bddb-5eca40e6e2b4 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:33.164647 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0323aa-b09e-4fe8-acda-809031372961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:33.165369 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0323aa-b09e-4fe8-acda-809031372961 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:33.165746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0323aa-b09e-4fe8-acda-809031372961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:33.166086 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0323aa-b09e-4fe8-acda-809031372961 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:33.166879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0323aa-b09e-4fe8-acda-809031372961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:33.168555 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0323aa-b09e-4fe8-acda-809031372961 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:33.179644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1078/2138] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:37:33.183791 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fd9e1-6e38-45d9-9552-4af69056c91e admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:33.193803 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fd9e1-6e38-45d9-9552-4af69056c91e admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:33.198353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0233edb-294f-4590-a7b7-5a35fd59864c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:33.199004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0233edb-294f-4590-a7b7-5a35fd59864c admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:33.199279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0233edb-294f-4590-a7b7-5a35fd59864c admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:33.199550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0233edb-294f-4590-a7b7-5a35fd59864c admin admin] PATH_INFO: `/v3/projects/f95c1b7268174fce821c7d056058dcde/users/06819823875146a99a05cc9b0bfe67ce/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:33.214891 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0233edb-294f-4590-a7b7-5a35fd59864c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f95c1b7268174fce821c7d056058dcde, user_id=06819823875146a99a05cc9b0bfe67ce, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:33.218377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0233edb-294f-4590-a7b7-5a35fd59864c admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:33.232009 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1061/2139] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:37:33 2022] PUT /identity/v3/projects/f95c1b7268174fce821c7d056058dcde/users/06819823875146a99a05cc9b0bfe67ce/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:33.243474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2c26c-6804-467c-ab1d-896802322490 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:33.243815 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2c26c-6804-467c-ab1d-896802322490 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:33.244100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2c26c-6804-467c-ab1d-896802322490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:33.277645 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-f7f2c26c-6804-467c-ab1d-896802322490 None None] MFA Rules not processed for user `06819823875146a99a05cc9b0bfe67ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:33.278918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f7f2c26c-6804-467c-ab1d-896802322490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:33.341021 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1079/2140] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:33 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:33.352448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0233edb-294f-4590-a7b7-5a35fd59864c admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:33.354193 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d0233edb-294f-4590-a7b7-5a35fd59864c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:33.364077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:33.432296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0233edb-294f-4590-a7b7-5a35fd59864c admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:33.436460 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023edd8-e4ce-4c77-8c0d-45470681cbc9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c50c2d1781b04a5fa8be81bce9205747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:33.437084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023edd8-e4ce-4c77-8c0d-45470681cbc9 service glance] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:33.437368 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023edd8-e4ce-4c77-8c0d-45470681cbc9 service glance] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:33.437638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023edd8-e4ce-4c77-8c0d-45470681cbc9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:33.445686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023edd8-e4ce-4c77-8c0d-45470681cbc9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:33.447107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023edd8-e4ce-4c77-8c0d-45470681cbc9 service glance] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:33.456314 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1062/2141] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:37:33 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:37:43.337801 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:43.452463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=32 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:49.522355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2c26c-6804-467c-ab1d-896802322490 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:49.523923 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f7f2c26c-6804-467c-ab1d-896802322490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:49.596749 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2c26c-6804-467c-ab1d-896802322490 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:49.601425 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f67420-0c96-40d2-8e2b-26d1615e9de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:49.602230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f67420-0c96-40d2-8e2b-26d1615e9de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:49.602518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f67420-0c96-40d2-8e2b-26d1615e9de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:49.602793 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f67420-0c96-40d2-8e2b-26d1615e9de5 admin admin] PATH_INFO: `/v3/users/287ffcd2b7df439c9e5a4dc65535eb0d` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:49.603975 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f67420-0c96-40d2-8e2b-26d1615e9de5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=287ffcd2b7df439c9e5a4dc65535eb0d)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:49.605960 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f67420-0c96-40d2-8e2b-26d1615e9de5 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:49.661353 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a2f67420-0c96-40d2-8e2b-26d1615e9de5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '287ffcd2b7df439c9e5a4dc65535eb0d'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:49.686747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a2f67420-0c96-40d2-8e2b-26d1615e9de5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '287ffcd2b7df439c9e5a4dc65535eb0d'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:49.708287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a2f67420-0c96-40d2-8e2b-26d1615e9de5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '287ffcd2b7df439c9e5a4dc65535eb0d'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:49.719062 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a2f67420-0c96-40d2-8e2b-26d1615e9de5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '287ffcd2b7df439c9e5a4dc65535eb0d'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:49.719862 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a2f67420-0c96-40d2-8e2b-26d1615e9de5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '287ffcd2b7df439c9e5a4dc65535eb0d'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:49.729293 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1080/2142] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:37:49 2022] DELETE /identity/v3/users/287ffcd2b7df439c9e5a4dc65535eb0d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:49.730986 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a3115-e84e-4610-954e-385bb6315678 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:49.731480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a3115-e84e-4610-954e-385bb6315678 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:49.731584 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a3115-e84e-4610-954e-385bb6315678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:49.753915 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-9e1a3115-e84e-4610-954e-385bb6315678 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:49.755223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9e1a3115-e84e-4610-954e-385bb6315678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:49.825347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1063/2143] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:49.835130 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f67420-0c96-40d2-8e2b-26d1615e9de5 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:49.838331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-a2f67420-0c96-40d2-8e2b-26d1615e9de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:49.901634 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f67420-0c96-40d2-8e2b-26d1615e9de5 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:49.907331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfd49f-e8be-4262-832b-918353cef9da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:49.908119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfd49f-e8be-4262-832b-918353cef9da service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:49.908569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfd49f-e8be-4262-832b-918353cef9da service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:49.908851 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfd49f-e8be-4262-832b-918353cef9da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:49.919732 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbfd49f-e8be-4262-832b-918353cef9da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:49.921266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbfd49f-e8be-4262-832b-918353cef9da service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:49.934574 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1081/2144] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:37:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:37:50.476578 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a3115-e84e-4610-954e-385bb6315678 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:50.478231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9e1a3115-e84e-4610-954e-385bb6315678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:50.502692 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a3115-e84e-4610-954e-385bb6315678 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:50.514477 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c488262-1b8a-405b-817b-beb395d97f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:50.515239 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c488262-1b8a-405b-817b-beb395d97f35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:50.515737 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c488262-1b8a-405b-817b-beb395d97f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:50.516246 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c488262-1b8a-405b-817b-beb395d97f35 admin admin] PATH_INFO: `/v3/projects/bd543689d3ca415fb6cbb077489d07f3` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:50.519832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c488262-1b8a-405b-817b-beb395d97f35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd543689d3ca415fb6cbb077489d07f3)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:50.521210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c488262-1b8a-405b-817b-beb395d97f35 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:50.581194 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0c488262-1b8a-405b-817b-beb395d97f35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd543689d3ca415fb6cbb077489d07f3', 'request_id': 'req-0c488262-1b8a-405b-817b-beb395d97f35'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:50.587021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0c488262-1b8a-405b-817b-beb395d97f35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd543689d3ca415fb6cbb077489d07f3', 'request_id': 'req-0c488262-1b8a-405b-817b-beb395d97f35'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:50.595675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0c488262-1b8a-405b-817b-beb395d97f35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd543689d3ca415fb6cbb077489d07f3', 'request_id': 'req-0c488262-1b8a-405b-817b-beb395d97f35'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:50.623781 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0c488262-1b8a-405b-817b-beb395d97f35 admin admin] The token cache is being invalidate because project bd543689d3ca415fb6cbb077489d07f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:37:50.624073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0c488262-1b8a-405b-817b-beb395d97f35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:50.624837 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0c488262-1b8a-405b-817b-beb395d97f35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:50.626636 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1064/2145] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:37:50 2022] DELETE /identity/v3/projects/bd543689d3ca415fb6cbb077489d07f3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:50.631486 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc69ed5-dbd8-4a7b-8133-1a9abaa209b9 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:50.631985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc69ed5-dbd8-4a7b-8133-1a9abaa209b9 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:50.632098 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc69ed5-dbd8-4a7b-8133-1a9abaa209b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:50.655917 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-2cc69ed5-dbd8-4a7b-8133-1a9abaa209b9 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:50.657222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-2cc69ed5-dbd8-4a7b-8133-1a9abaa209b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:50.677039 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:50.735611 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1082/2146] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:50.743814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72060a2e-7773-4eed-bd43-2655c110486d None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:50.744381 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72060a2e-7773-4eed-bd43-2655c110486d None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:50.744534 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72060a2e-7773-4eed-bd43-2655c110486d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:50.764915 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-72060a2e-7773-4eed-bd43-2655c110486d None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:50.766115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-72060a2e-7773-4eed-bd43-2655c110486d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:50.773825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1065/2147] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:50.778110 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc69ed5-dbd8-4a7b-8133-1a9abaa209b9 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:50.790072 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc69ed5-dbd8-4a7b-8133-1a9abaa209b9 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:50.794554 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0c701-b798-4d27-99ae-ad0f805989a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:50.795177 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0c701-b798-4d27-99ae-ad0f805989a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:50.795455 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0c701-b798-4d27-99ae-ad0f805989a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:50.795727 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0c701-b798-4d27-99ae-ad0f805989a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:50.796418 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0c701-b798-4d27-99ae-ad0f805989a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:37:50.796720 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0c701-b798-4d27-99ae-ad0f805989a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:50.798370 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0c701-b798-4d27-99ae-ad0f805989a7 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:50.812779 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1083/2148] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:37:50.817050 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72060a2e-7773-4eed-bd43-2655c110486d None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:50.826862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72060a2e-7773-4eed-bd43-2655c110486d None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:50.831807 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141c013-f1b8-4a5c-b676-6f240841a5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:50.832419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141c013-f1b8-4a5c-b676-6f240841a5e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:50.832704 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141c013-f1b8-4a5c-b676-6f240841a5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:50.833010 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141c013-f1b8-4a5c-b676-6f240841a5e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:50.833679 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141c013-f1b8-4a5c-b676-6f240841a5e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:50.835571 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141c013-f1b8-4a5c-b676-6f240841a5e2 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:50.865460 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1066/2149] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:37:50 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:37:50.869732 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0c701-b798-4d27-99ae-ad0f805989a7 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:50.880464 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0c701-b798-4d27-99ae-ad0f805989a7 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:50.938087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa2096-9775-43d8-a383-9e93601daae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:50.938794 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa2096-9775-43d8-a383-9e93601daae1 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:50.939160 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa2096-9775-43d8-a383-9e93601daae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:50.939434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa2096-9775-43d8-a383-9e93601daae1 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:50.940251 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa2096-9775-43d8-a383-9e93601daae1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:50.942387 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa2096-9775-43d8-a383-9e93601daae1 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:50.966882 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1084/2150] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:37:50 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:37:50.968818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141c013-f1b8-4a5c-b676-6f240841a5e2 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:50.979390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141c013-f1b8-4a5c-b676-6f240841a5e2 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:50.985640 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080a0a14-b508-4114-a307-7b00b54ab3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:50.986370 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080a0a14-b508-4114-a307-7b00b54ab3df admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:50.986727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080a0a14-b508-4114-a307-7b00b54ab3df admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:50.987077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080a0a14-b508-4114-a307-7b00b54ab3df admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:50.987852 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080a0a14-b508-4114-a307-7b00b54ab3df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:50.989626 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080a0a14-b508-4114-a307-7b00b54ab3df admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:51.000956 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1067/2151] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:37:51.005893 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa2096-9775-43d8-a383-9e93601daae1 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:51.017493 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa2096-9775-43d8-a383-9e93601daae1 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:51.022839 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54a57f-6c61-4d30-955f-81939889d117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:51.024150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54a57f-6c61-4d30-955f-81939889d117 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:51.024150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54a57f-6c61-4d30-955f-81939889d117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:51.024459 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54a57f-6c61-4d30-955f-81939889d117 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:51.025419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54a57f-6c61-4d30-955f-81939889d117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:51.027241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54a57f-6c61-4d30-955f-81939889d117 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:51.038983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1085/2152] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:37:51.043216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080a0a14-b508-4114-a307-7b00b54ab3df admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:51.054315 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080a0a14-b508-4114-a307-7b00b54ab3df admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:51.059364 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fb4c8-6007-40f3-b260-182219883158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:51.060584 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fb4c8-6007-40f3-b260-182219883158 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:51.060868 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fb4c8-6007-40f3-b260-182219883158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:51.061184 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fb4c8-6007-40f3-b260-182219883158 admin admin] PATH_INFO: `/v3/projects/a240eaa073f3472a8ec5f6f6db94cb87/users/1a90c81c578e49f9aae722dfb2f2e8d4/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:51.076197 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5fb4c8-6007-40f3-b260-182219883158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a240eaa073f3472a8ec5f6f6db94cb87, user_id=1a90c81c578e49f9aae722dfb2f2e8d4, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:51.078923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5fb4c8-6007-40f3-b260-182219883158 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:51.092596 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1068/2153] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:37:51 2022] PUT /identity/v3/projects/a240eaa073f3472a8ec5f6f6db94cb87/users/1a90c81c578e49f9aae722dfb2f2e8d4/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:37:51.103839 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5388a-6160-46c2-9069-e6c0b4296ade None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:51.104123 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5388a-6160-46c2-9069-e6c0b4296ade None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:51.104395 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5388a-6160-46c2-9069-e6c0b4296ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:51.134932 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-01c5388a-6160-46c2-9069-e6c0b4296ade None None] MFA Rules not processed for user `1a90c81c578e49f9aae722dfb2f2e8d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:51.136136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-01c5388a-6160-46c2-9069-e6c0b4296ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:51.231917 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1086/2154] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:51.243011 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fb4c8-6007-40f3-b260-182219883158 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:51.244772 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ac5fb4c8-6007-40f3-b260-182219883158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:51.267265 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:51.306547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fb4c8-6007-40f3-b260-182219883158 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:51.310408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afaf94-bddf-4cae-a08e-eed1329d0f87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:51.311091 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afaf94-bddf-4cae-a08e-eed1329d0f87 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:51.311388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afaf94-bddf-4cae-a08e-eed1329d0f87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:51.311672 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afaf94-bddf-4cae-a08e-eed1329d0f87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:51.320823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afaf94-bddf-4cae-a08e-eed1329d0f87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:51.322289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afaf94-bddf-4cae-a08e-eed1329d0f87 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:51.331315 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1069/2155] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:37:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:37:59.622352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5388a-6160-46c2-9069-e6c0b4296ade None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:59.641634 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:37:59.693477 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5388a-6160-46c2-9069-e6c0b4296ade None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:59.697989 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96504c56-0bda-4bac-b79f-b4bfbedbe767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:59.698629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96504c56-0bda-4bac-b79f-b4bfbedbe767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:59.698911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96504c56-0bda-4bac-b79f-b4bfbedbe767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:59.699187 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96504c56-0bda-4bac-b79f-b4bfbedbe767 admin admin] PATH_INFO: `/v3/users/1a90c81c578e49f9aae722dfb2f2e8d4` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:59.701001 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96504c56-0bda-4bac-b79f-b4bfbedbe767 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a90c81c578e49f9aae722dfb2f2e8d4)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:59.702925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96504c56-0bda-4bac-b79f-b4bfbedbe767 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:59.748304 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-96504c56-0bda-4bac-b79f-b4bfbedbe767 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a90c81c578e49f9aae722dfb2f2e8d4'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:59.755400 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-96504c56-0bda-4bac-b79f-b4bfbedbe767 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a90c81c578e49f9aae722dfb2f2e8d4'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:59.768145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-96504c56-0bda-4bac-b79f-b4bfbedbe767 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a90c81c578e49f9aae722dfb2f2e8d4'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:59.774844 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-96504c56-0bda-4bac-b79f-b4bfbedbe767 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a90c81c578e49f9aae722dfb2f2e8d4'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:59.775484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-96504c56-0bda-4bac-b79f-b4bfbedbe767 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a90c81c578e49f9aae722dfb2f2e8d4'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:37:59.781311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1087/2156] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:37:59 2022] DELETE /identity/v3/users/1a90c81c578e49f9aae722dfb2f2e8d4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:37:59.785739 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29825c-6282-4148-9ebd-2dc2469daeb2 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:59.786199 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29825c-6282-4148-9ebd-2dc2469daeb2 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:59.786332 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29825c-6282-4148-9ebd-2dc2469daeb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:59.804918 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-4f29825c-6282-4148-9ebd-2dc2469daeb2 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:37:59.806138 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-4f29825c-6282-4148-9ebd-2dc2469daeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:59.866628 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1070/2157] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:37:59.876213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96504c56-0bda-4bac-b79f-b4bfbedbe767 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:37:59.877606 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-96504c56-0bda-4bac-b79f-b4bfbedbe767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:37:59.934510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96504c56-0bda-4bac-b79f-b4bfbedbe767 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:37:59.938310 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30621707-3c36-4b03-b3e1-245a7b7bde8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:37:59.938916 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30621707-3c36-4b03-b3e1-245a7b7bde8c service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:37:59.939194 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30621707-3c36-4b03-b3e1-245a7b7bde8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:37:59.939465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30621707-3c36-4b03-b3e1-245a7b7bde8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:37:59.948361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30621707-3c36-4b03-b3e1-245a7b7bde8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:37:59.949819 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30621707-3c36-4b03-b3e1-245a7b7bde8c service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:37:59.959546 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1088/2158] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:37:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:38:00.230167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29825c-6282-4148-9ebd-2dc2469daeb2 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:00.232537 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-4f29825c-6282-4148-9ebd-2dc2469daeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:00.245839 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29825c-6282-4148-9ebd-2dc2469daeb2 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:00.251698 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c2f2a-3383-40a4-beb9-560d4a7af547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:00.252204 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c2f2a-3383-40a4-beb9-560d4a7af547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:00.252488 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c2f2a-3383-40a4-beb9-560d4a7af547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:00.252761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c2f2a-3383-40a4-beb9-560d4a7af547 admin admin] PATH_INFO: `/v3/projects/a240eaa073f3472a8ec5f6f6db94cb87` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:00.253994 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c2f2a-3383-40a4-beb9-560d4a7af547 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a240eaa073f3472a8ec5f6f6db94cb87)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:00.256335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c2f2a-3383-40a4-beb9-560d4a7af547 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:00.314656 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-760c2f2a-3383-40a4-beb9-560d4a7af547 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a240eaa073f3472a8ec5f6f6db94cb87', 'request_id': 'req-760c2f2a-3383-40a4-beb9-560d4a7af547'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:00.322424 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-760c2f2a-3383-40a4-beb9-560d4a7af547 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a240eaa073f3472a8ec5f6f6db94cb87', 'request_id': 'req-760c2f2a-3383-40a4-beb9-560d4a7af547'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:00.331386 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d22ed-951d-4c56-8f33-86f982bcbe57 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:00.331923 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d22ed-951d-4c56-8f33-86f982bcbe57 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:00.332375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d22ed-951d-4c56-8f33-86f982bcbe57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:00.337608 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-760c2f2a-3383-40a4-beb9-560d4a7af547 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a240eaa073f3472a8ec5f6f6db94cb87', 'request_id': 'req-760c2f2a-3383-40a4-beb9-560d4a7af547'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:00.351461 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-760c2f2a-3383-40a4-beb9-560d4a7af547 admin admin] The token cache is being invalidate because project a240eaa073f3472a8ec5f6f6db94cb87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:38:00.351753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-760c2f2a-3383-40a4-beb9-560d4a7af547 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:00.352557 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-760c2f2a-3383-40a4-beb9-560d4a7af547 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:00.354561 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1071/2159] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:38:00 2022] DELETE /identity/v3/projects/a240eaa073f3472a8ec5f6f6db94cb87 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:38:00.356562 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-143d22ed-951d-4c56-8f33-86f982bcbe57 None None] MFA Rules not processed for user `07d6bffc69674103b4c6540598a780b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:00.358038 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-143d22ed-951d-4c56-8f33-86f982bcbe57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:00.422048 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:00.448109 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1089/2160] 10.222.0.74 () {58 vars in 1009 bytes} [Wed Jul 27 16:38:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:38:00.512043 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c2f2a-3383-40a4-beb9-560d4a7af547 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:00.513672 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-760c2f2a-3383-40a4-beb9-560d4a7af547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:00.569292 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:00.597445 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c2f2a-3383-40a4-beb9-560d4a7af547 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:00.602751 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a029b8-7cc5-4f6c-a83c-9d995bf4eba2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb45ded9f75e4fa2973b564f83bdfaf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:00.603377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a029b8-7cc5-4f6c-a83c-9d995bf4eba2 service swift] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:00.603683 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a029b8-7cc5-4f6c-a83c-9d995bf4eba2 service swift] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:00.603961 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a029b8-7cc5-4f6c-a83c-9d995bf4eba2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:00.614484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a029b8-7cc5-4f6c-a83c-9d995bf4eba2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:00.616081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a029b8-7cc5-4f6c-a83c-9d995bf4eba2 service swift] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:00.625735 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1072/2161] 10.222.0.74 () {60 vars in 1384 bytes} [Wed Jul 27 16:38:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 16:38:01.828329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d22ed-951d-4c56-8f33-86f982bcbe57 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:01.829918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-143d22ed-951d-4c56-8f33-86f982bcbe57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:01.904372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d22ed-951d-4c56-8f33-86f982bcbe57 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:01.915785 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18053c-e5d7-4d29-88b0-cecf4ee6cee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:01.916626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18053c-e5d7-4d29-88b0-cecf4ee6cee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:01.917265 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18053c-e5d7-4d29-88b0-cecf4ee6cee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:01.918075 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18053c-e5d7-4d29-88b0-cecf4ee6cee5 admin admin] PATH_INFO: `/v3/users/13b5c9d74a4f4fc99f0be66e47e53c7a` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:01.921575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e18053c-e5d7-4d29-88b0-cecf4ee6cee5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13b5c9d74a4f4fc99f0be66e47e53c7a)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:01.925026 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e18053c-e5d7-4d29-88b0-cecf4ee6cee5 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:01.995695 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-5e18053c-e5d7-4d29-88b0-cecf4ee6cee5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13b5c9d74a4f4fc99f0be66e47e53c7a'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:02.006005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-5e18053c-e5d7-4d29-88b0-cecf4ee6cee5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13b5c9d74a4f4fc99f0be66e47e53c7a'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:02.030115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-5e18053c-e5d7-4d29-88b0-cecf4ee6cee5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13b5c9d74a4f4fc99f0be66e47e53c7a'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:02.038210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-5e18053c-e5d7-4d29-88b0-cecf4ee6cee5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13b5c9d74a4f4fc99f0be66e47e53c7a'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:02.039014 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-5e18053c-e5d7-4d29-88b0-cecf4ee6cee5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13b5c9d74a4f4fc99f0be66e47e53c7a'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:02.047817 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1090/2162] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:38:01 2022] DELETE /identity/v3/users/13b5c9d74a4f4fc99f0be66e47e53c7a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:38:02.052643 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af21725-3705-4324-a276-f16705c85690 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:02.053131 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af21725-3705-4324-a276-f16705c85690 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:02.053221 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af21725-3705-4324-a276-f16705c85690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:02.073661 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-2af21725-3705-4324-a276-f16705c85690 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:02.075019 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-2af21725-3705-4324-a276-f16705c85690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:02.244462 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1073/2163] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:38:02.255953 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18053c-e5d7-4d29-88b0-cecf4ee6cee5 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:02.258251 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-5e18053c-e5d7-4d29-88b0-cecf4ee6cee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:02.321750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18053c-e5d7-4d29-88b0-cecf4ee6cee5 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:02.325697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c254e0-a904-4fe9-bfcb-5547eb0b279d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:02.326315 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c254e0-a904-4fe9-bfcb-5547eb0b279d service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:02.326591 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c254e0-a904-4fe9-bfcb-5547eb0b279d service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:02.326863 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c254e0-a904-4fe9-bfcb-5547eb0b279d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:02.339125 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c254e0-a904-4fe9-bfcb-5547eb0b279d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:02.340856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c254e0-a904-4fe9-bfcb-5547eb0b279d service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:02.350318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1091/2164] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:38:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:38:02.662142 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af21725-3705-4324-a276-f16705c85690 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:02.664388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-2af21725-3705-4324-a276-f16705c85690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:02.681732 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af21725-3705-4324-a276-f16705c85690 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:02.687434 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b205f-c8fb-4a7d-b150-d90ee3d2d1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:02.688217 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b205f-c8fb-4a7d-b150-d90ee3d2d1a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:02.688484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b205f-c8fb-4a7d-b150-d90ee3d2d1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:02.689019 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b205f-c8fb-4a7d-b150-d90ee3d2d1a5 admin admin] PATH_INFO: `/v3/projects/0de76c9908a24002a9a5ae43249489f8` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:02.690026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b205f-c8fb-4a7d-b150-d90ee3d2d1a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0de76c9908a24002a9a5ae43249489f8)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:02.691945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b205f-c8fb-4a7d-b150-d90ee3d2d1a5 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:02.749204 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-758b205f-c8fb-4a7d-b150-d90ee3d2d1a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0de76c9908a24002a9a5ae43249489f8', 'request_id': 'req-758b205f-c8fb-4a7d-b150-d90ee3d2d1a5'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:02.755557 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-758b205f-c8fb-4a7d-b150-d90ee3d2d1a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0de76c9908a24002a9a5ae43249489f8', 'request_id': 'req-758b205f-c8fb-4a7d-b150-d90ee3d2d1a5'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:02.765240 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-758b205f-c8fb-4a7d-b150-d90ee3d2d1a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0de76c9908a24002a9a5ae43249489f8', 'request_id': 'req-758b205f-c8fb-4a7d-b150-d90ee3d2d1a5'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:02.777767 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-758b205f-c8fb-4a7d-b150-d90ee3d2d1a5 admin admin] The token cache is being invalidate because project 0de76c9908a24002a9a5ae43249489f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:38:02.778593 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-758b205f-c8fb-4a7d-b150-d90ee3d2d1a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:02.779230 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-758b205f-c8fb-4a7d-b150-d90ee3d2d1a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:02.780990 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1074/2165] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:38:02 2022] DELETE /identity/v3/projects/0de76c9908a24002a9a5ae43249489f8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:38:02.784954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c254e0-a904-4fe9-bfcb-5547eb0b279d service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:02.795676 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c254e0-a904-4fe9-bfcb-5547eb0b279d service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:02.800056 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:02.861905 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe2b15-6168-46cb-9988-0b3ea992218a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:02.862573 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe2b15-6168-46cb-9988-0b3ea992218a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:02.862862 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe2b15-6168-46cb-9988-0b3ea992218a admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:02.863138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe2b15-6168-46cb-9988-0b3ea992218a admin admin] PATH_INFO: `/v3/users/6c1556baf8974cbfaab5cfcefbb1acd9` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:02.864540 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe2b15-6168-46cb-9988-0b3ea992218a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c1556baf8974cbfaab5cfcefbb1acd9)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:02.867311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe2b15-6168-46cb-9988-0b3ea992218a admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:02.935745 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-c1fe2b15-6168-46cb-9988-0b3ea992218a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c1556baf8974cbfaab5cfcefbb1acd9'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:02.945000 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-c1fe2b15-6168-46cb-9988-0b3ea992218a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c1556baf8974cbfaab5cfcefbb1acd9'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:02.958608 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-c1fe2b15-6168-46cb-9988-0b3ea992218a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c1556baf8974cbfaab5cfcefbb1acd9'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:02.967863 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-c1fe2b15-6168-46cb-9988-0b3ea992218a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c1556baf8974cbfaab5cfcefbb1acd9'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:02.968869 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-c1fe2b15-6168-46cb-9988-0b3ea992218a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c1556baf8974cbfaab5cfcefbb1acd9'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:02.975196 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1092/2166] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:38:02 2022] DELETE /identity/v3/users/6c1556baf8974cbfaab5cfcefbb1acd9 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:38:03.406202 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b205f-c8fb-4a7d-b150-d90ee3d2d1a5 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:03.408669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-758b205f-c8fb-4a7d-b150-d90ee3d2d1a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:03.425544 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b205f-c8fb-4a7d-b150-d90ee3d2d1a5 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:03.546630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5060f1-304c-46fa-8ed6-0042b0088aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:03.547569 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5060f1-304c-46fa-8ed6-0042b0088aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:03.547967 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5060f1-304c-46fa-8ed6-0042b0088aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:03.548253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5060f1-304c-46fa-8ed6-0042b0088aee admin admin] PATH_INFO: `/v3/projects/0928b23755fc4186b04c65dade0f7f20` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:03.549469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5060f1-304c-46fa-8ed6-0042b0088aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=0928b23755fc4186b04c65dade0f7f20)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:03.551383 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5060f1-304c-46fa-8ed6-0042b0088aee admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:03.553805 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:03.606889 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-4a5060f1-304c-46fa-8ed6-0042b0088aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0928b23755fc4186b04c65dade0f7f20', 'request_id': 'req-4a5060f1-304c-46fa-8ed6-0042b0088aee'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:03.613624 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-4a5060f1-304c-46fa-8ed6-0042b0088aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0928b23755fc4186b04c65dade0f7f20', 'request_id': 'req-4a5060f1-304c-46fa-8ed6-0042b0088aee'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:03.626265 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-4a5060f1-304c-46fa-8ed6-0042b0088aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0928b23755fc4186b04c65dade0f7f20', 'request_id': 'req-4a5060f1-304c-46fa-8ed6-0042b0088aee'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:03.643154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-4a5060f1-304c-46fa-8ed6-0042b0088aee admin admin] The token cache is being invalidate because project 0928b23755fc4186b04c65dade0f7f20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:38:03.643673 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-4a5060f1-304c-46fa-8ed6-0042b0088aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:03.644626 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-4a5060f1-304c-46fa-8ed6-0042b0088aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:03.646815 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1075/2167] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:38:03 2022] DELETE /identity/v3/projects/0928b23755fc4186b04c65dade0f7f20 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:38:03.656085 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44009922-4846-40d2-b4f0-c4fc48757f03 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:03.656579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44009922-4846-40d2-b4f0-c4fc48757f03 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:03.657262 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44009922-4846-40d2-b4f0-c4fc48757f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:03.683535 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-44009922-4846-40d2-b4f0-c4fc48757f03 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:03.685364 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-44009922-4846-40d2-b4f0-c4fc48757f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:03.776254 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1093/2168] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:38:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:38:03.784291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b1fcc-9377-4cef-9ecb-68c464642894 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:03.784821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b1fcc-9377-4cef-9ecb-68c464642894 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:03.785332 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b1fcc-9377-4cef-9ecb-68c464642894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:03.814358 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-3a0b1fcc-9377-4cef-9ecb-68c464642894 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:03.815995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-3a0b1fcc-9377-4cef-9ecb-68c464642894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:03.825317 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1076/2169] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:38:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:38:03.830570 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44009922-4846-40d2-b4f0-c4fc48757f03 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:03.844140 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44009922-4846-40d2-b4f0-c4fc48757f03 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:03.862263 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b3ab2-61bb-4fb2-b736-b609b564250a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:03.862263 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b3ab2-61bb-4fb2-b736-b609b564250a admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:03.862263 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b3ab2-61bb-4fb2-b736-b609b564250a admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:03.862263 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b3ab2-61bb-4fb2-b736-b609b564250a admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:03.863867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271b3ab2-61bb-4fb2-b736-b609b564250a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:38:03.863867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271b3ab2-61bb-4fb2-b736-b609b564250a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:03.863867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271b3ab2-61bb-4fb2-b736-b609b564250a admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:03.876866 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1094/2170] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:38:03.881330 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b1fcc-9377-4cef-9ecb-68c464642894 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:03.890674 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b1fcc-9377-4cef-9ecb-68c464642894 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:03.895307 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcebf43-c5a0-4630-b888-a8be637e9984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:03.895911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcebf43-c5a0-4630-b888-a8be637e9984 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:03.896191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcebf43-c5a0-4630-b888-a8be637e9984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:03.896466 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcebf43-c5a0-4630-b888-a8be637e9984 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:03.897145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcebf43-c5a0-4630-b888-a8be637e9984 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:03.899086 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcebf43-c5a0-4630-b888-a8be637e9984 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:03.928487 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1077/2171] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:38:03 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:38:03.932677 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b3ab2-61bb-4fb2-b736-b609b564250a admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:03.943715 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b3ab2-61bb-4fb2-b736-b609b564250a admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:03.952993 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:04.026405 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be535d-5011-4517-bb0f-a7893c35463c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:04.027093 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be535d-5011-4517-bb0f-a7893c35463c admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:04.027405 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be535d-5011-4517-bb0f-a7893c35463c admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:04.027687 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be535d-5011-4517-bb0f-a7893c35463c admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:04.029065 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be535d-5011-4517-bb0f-a7893c35463c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:04.031171 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be535d-5011-4517-bb0f-a7893c35463c admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:04.057382 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1095/2172] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:38:03 2022] POST /identity/v3/users => generated 294 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:38:04.061901 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcebf43-c5a0-4630-b888-a8be637e9984 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:04.076040 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcebf43-c5a0-4630-b888-a8be637e9984 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:04.082141 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da52ab-e501-4b0c-a9ca-b19314d1a2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:04.082886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da52ab-e501-4b0c-a9ca-b19314d1a2bb admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:04.083241 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da52ab-e501-4b0c-a9ca-b19314d1a2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:04.083519 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da52ab-e501-4b0c-a9ca-b19314d1a2bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:04.084304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da52ab-e501-4b0c-a9ca-b19314d1a2bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:04.086018 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da52ab-e501-4b0c-a9ca-b19314d1a2bb admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:04.101070 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1078/2173] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:38:04.102408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be535d-5011-4517-bb0f-a7893c35463c admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:04.113894 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be535d-5011-4517-bb0f-a7893c35463c admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:04.119351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533fa19-8eb7-4790-a728-b984ed747e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:04.119970 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533fa19-8eb7-4790-a728-b984ed747e45 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:04.120253 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533fa19-8eb7-4790-a728-b984ed747e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:04.120531 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533fa19-8eb7-4790-a728-b984ed747e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:04.121269 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533fa19-8eb7-4790-a728-b984ed747e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:04.122875 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533fa19-8eb7-4790-a728-b984ed747e45 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:04.134225 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1096/2174] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:38:04.138475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da52ab-e501-4b0c-a9ca-b19314d1a2bb admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:04.157083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da52ab-e501-4b0c-a9ca-b19314d1a2bb admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:04.162670 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0804e52-0765-4f30-908b-2b4568d68fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:04.163651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0804e52-0765-4f30-908b-2b4568d68fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:04.163925 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0804e52-0765-4f30-908b-2b4568d68fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:04.164223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0804e52-0765-4f30-908b-2b4568d68fb1 admin admin] PATH_INFO: `/v3/projects/2d57a50bd11642e19286b07af246109a/users/7ea6d04cabf14f81a5f82115e69465ca/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:04.181291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0804e52-0765-4f30-908b-2b4568d68fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d57a50bd11642e19286b07af246109a, user_id=7ea6d04cabf14f81a5f82115e69465ca, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:04.185349 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0804e52-0765-4f30-908b-2b4568d68fb1 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:04.199126 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1079/2175] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:38:04 2022] PUT /identity/v3/projects/2d57a50bd11642e19286b07af246109a/users/7ea6d04cabf14f81a5f82115e69465ca/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:38:04.211063 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf451c-19a6-4b01-811e-05acac7bbcd2 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:04.211354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf451c-19a6-4b01-811e-05acac7bbcd2 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:04.211631 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf451c-19a6-4b01-811e-05acac7bbcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:04.242233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-a9cf451c-19a6-4b01-811e-05acac7bbcd2 None None] MFA Rules not processed for user `7ea6d04cabf14f81a5f82115e69465ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:04.243662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-a9cf451c-19a6-4b01-811e-05acac7bbcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:04.338440 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1097/2176] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:38:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:38:04.343373 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0804e52-0765-4f30-908b-2b4568d68fb1 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:04.432640 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0804e52-0765-4f30-908b-2b4568d68fb1 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:04.440843 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d59be6-07ef-4370-b2ec-b06ecddc9eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:04.441645 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d59be6-07ef-4370-b2ec-b06ecddc9eab admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:04.442033 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d59be6-07ef-4370-b2ec-b06ecddc9eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:04.442427 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d59be6-07ef-4370-b2ec-b06ecddc9eab admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:04.443238 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d59be6-07ef-4370-b2ec-b06ecddc9eab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:04.445510 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d59be6-07ef-4370-b2ec-b06ecddc9eab admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:04.482387 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1080/2177] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:38:04 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:38:04.486075 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf451c-19a6-4b01-811e-05acac7bbcd2 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:04.505798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf451c-19a6-4b01-811e-05acac7bbcd2 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:04.586565 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a2ba4-c64c-4408-bcf3-7aa0e70461be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:04.587283 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a2ba4-c64c-4408-bcf3-7aa0e70461be admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:04.587645 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a2ba4-c64c-4408-bcf3-7aa0e70461be admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:04.588593 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a2ba4-c64c-4408-bcf3-7aa0e70461be admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:04.589414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a2ba4-c64c-4408-bcf3-7aa0e70461be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:04.592364 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a2ba4-c64c-4408-bcf3-7aa0e70461be admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:04.629038 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1098/2178] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:38:04 2022] POST /identity/v3/users => generated 299 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:38:04.630271 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d59be6-07ef-4370-b2ec-b06ecddc9eab admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:04.643397 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d59be6-07ef-4370-b2ec-b06ecddc9eab admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:04.648390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e5897-8011-4eb4-b145-2262bcec18d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:04.649109 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e5897-8011-4eb4-b145-2262bcec18d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:04.649511 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e5897-8011-4eb4-b145-2262bcec18d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:04.649700 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e5897-8011-4eb4-b145-2262bcec18d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:04.650337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e5897-8011-4eb4-b145-2262bcec18d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:04.652099 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e5897-8011-4eb4-b145-2262bcec18d3 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:04.662036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:04.676699 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1081/2179] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:38:04.678615 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a2ba4-c64c-4408-bcf3-7aa0e70461be admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:04.690206 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a2ba4-c64c-4408-bcf3-7aa0e70461be admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:04.695516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20579a8d-52ed-4ca8-bdee-9f4635253f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:04.696218 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20579a8d-52ed-4ca8-bdee-9f4635253f19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:04.696689 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20579a8d-52ed-4ca8-bdee-9f4635253f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:04.696807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20579a8d-52ed-4ca8-bdee-9f4635253f19 admin admin] PATH_INFO: `/v3/projects/91a8a870d9f14f56bd3d486115622992/users/47a150d586ed42e68d6a5d703b8805e1/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:04.707433 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:04.718512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20579a8d-52ed-4ca8-bdee-9f4635253f19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91a8a870d9f14f56bd3d486115622992, user_id=47a150d586ed42e68d6a5d703b8805e1, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:04.721353 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20579a8d-52ed-4ca8-bdee-9f4635253f19 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:04.736496 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1099/2180] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:38:04 2022] PUT /identity/v3/projects/91a8a870d9f14f56bd3d486115622992/users/47a150d586ed42e68d6a5d703b8805e1/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:38:04.740707 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e5897-8011-4eb4-b145-2262bcec18d3 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:04.753340 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e5897-8011-4eb4-b145-2262bcec18d3 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:04.837783 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a56c86-c8c2-4261-8843-b4359c10a160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:04.838517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a56c86-c8c2-4261-8843-b4359c10a160 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:04.838886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a56c86-c8c2-4261-8843-b4359c10a160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:04.839122 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a56c86-c8c2-4261-8843-b4359c10a160 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:04.840247 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a56c86-c8c2-4261-8843-b4359c10a160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:04.842618 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a56c86-c8c2-4261-8843-b4359c10a160 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:04.860017 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1082/2181] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:38:04.865156 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20579a8d-52ed-4ca8-bdee-9f4635253f19 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:04.880495 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20579a8d-52ed-4ca8-bdee-9f4635253f19 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:04.889241 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9968ba-1f18-4f78-bd1b-c604bdbb9849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:04.891163 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9968ba-1f18-4f78-bd1b-c604bdbb9849 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:04.892110 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9968ba-1f18-4f78-bd1b-c604bdbb9849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:04.892622 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9968ba-1f18-4f78-bd1b-c604bdbb9849 admin admin] PATH_INFO: `/v3/projects/91a8a870d9f14f56bd3d486115622992/users/47a150d586ed42e68d6a5d703b8805e1/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:04.895844 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9968ba-1f18-4f78-bd1b-c604bdbb9849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91a8a870d9f14f56bd3d486115622992, user_id=47a150d586ed42e68d6a5d703b8805e1, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:04.899652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9968ba-1f18-4f78-bd1b-c604bdbb9849 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:04.919793 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1100/2182] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:38:04 2022] PUT /identity/v3/projects/91a8a870d9f14f56bd3d486115622992/users/47a150d586ed42e68d6a5d703b8805e1/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:38:04.931112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80c97a-cc4b-4f79-bea5-8dbf6d2d3728 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:04.931779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80c97a-cc4b-4f79-bea5-8dbf6d2d3728 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:04.932692 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80c97a-cc4b-4f79-bea5-8dbf6d2d3728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:04.973256 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-4b80c97a-cc4b-4f79-bea5-8dbf6d2d3728 None None] MFA Rules not processed for user `47a150d586ed42e68d6a5d703b8805e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:04.975161 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-4b80c97a-cc4b-4f79-bea5-8dbf6d2d3728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:05.114341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1083/2183] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:38:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:38:05.125450 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9968ba-1f18-4f78-bd1b-c604bdbb9849 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:05.127564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-cb9968ba-1f18-4f78-bd1b-c604bdbb9849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:05.221811 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9968ba-1f18-4f78-bd1b-c604bdbb9849 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:05.225854 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221d740-4f88-4925-98ef-908c196cf4eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:05.226413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221d740-4f88-4925-98ef-908c196cf4eb service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:05.226704 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221d740-4f88-4925-98ef-908c196cf4eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:05.226976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221d740-4f88-4925-98ef-908c196cf4eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:05.237973 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221d740-4f88-4925-98ef-908c196cf4eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:05.239433 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221d740-4f88-4925-98ef-908c196cf4eb service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:05.251118 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1101/2184] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:38:05 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:38:05.402816 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80c97a-cc4b-4f79-bea5-8dbf6d2d3728 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:05.479763 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80c97a-cc4b-4f79-bea5-8dbf6d2d3728 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:05.484834 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c492d-6491-4868-b96d-800b4768d902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:05.485503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c492d-6491-4868-b96d-800b4768d902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:05.485909 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c492d-6491-4868-b96d-800b4768d902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:05.486182 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c492d-6491-4868-b96d-800b4768d902 admin admin] PATH_INFO: `/v3/users/7ea6d04cabf14f81a5f82115e69465ca` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:05.487370 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c492d-6491-4868-b96d-800b4768d902 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ea6d04cabf14f81a5f82115e69465ca)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:05.489387 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c492d-6491-4868-b96d-800b4768d902 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:05.514272 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:05.555912 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8f5c492d-6491-4868-b96d-800b4768d902 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ea6d04cabf14f81a5f82115e69465ca'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:05.567554 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8f5c492d-6491-4868-b96d-800b4768d902 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ea6d04cabf14f81a5f82115e69465ca'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:05.582954 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8f5c492d-6491-4868-b96d-800b4768d902 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ea6d04cabf14f81a5f82115e69465ca'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:05.599884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8f5c492d-6491-4868-b96d-800b4768d902 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ea6d04cabf14f81a5f82115e69465ca'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:05.600988 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8f5c492d-6491-4868-b96d-800b4768d902 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ea6d04cabf14f81a5f82115e69465ca'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:05.622694 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1084/2185] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:38:05 2022] DELETE /identity/v3/users/7ea6d04cabf14f81a5f82115e69465ca => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:38:05.624155 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fb5eb-636f-4d82-b118-990a1ade9849 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:05.624443 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fb5eb-636f-4d82-b118-990a1ade9849 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:05.624719 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fb5eb-636f-4d82-b118-990a1ade9849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:05.648563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-214fb5eb-636f-4d82-b118-990a1ade9849 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:05.649641 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-214fb5eb-636f-4d82-b118-990a1ade9849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:05.740840 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1102/2186] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:38:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:38:05.748374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c492d-6491-4868-b96d-800b4768d902 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:05.749885 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-8f5c492d-6491-4868-b96d-800b4768d902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:05.815309 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c492d-6491-4868-b96d-800b4768d902 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:05.821394 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b271f2-b48c-43f9-acff-d12e7850af85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:05.822130 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b271f2-b48c-43f9-acff-d12e7850af85 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:05.822560 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b271f2-b48c-43f9-acff-d12e7850af85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:05.822913 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b271f2-b48c-43f9-acff-d12e7850af85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:05.833278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b271f2-b48c-43f9-acff-d12e7850af85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:05.834912 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b271f2-b48c-43f9-acff-d12e7850af85 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:05.846448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1085/2187] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:38:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:38:06.215235 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fb5eb-636f-4d82-b118-990a1ade9849 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:06.218652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-214fb5eb-636f-4d82-b118-990a1ade9849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:06.253526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fb5eb-636f-4d82-b118-990a1ade9849 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:06.258931 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bb6c8-b4cc-4520-9d40-68f0b7c2657d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:06.259583 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bb6c8-b4cc-4520-9d40-68f0b7c2657d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:06.259890 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bb6c8-b4cc-4520-9d40-68f0b7c2657d admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:06.260137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bb6c8-b4cc-4520-9d40-68f0b7c2657d admin admin] PATH_INFO: `/v3/projects/2d57a50bd11642e19286b07af246109a` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:06.261324 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652bb6c8-b4cc-4520-9d40-68f0b7c2657d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d57a50bd11642e19286b07af246109a)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:06.264046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652bb6c8-b4cc-4520-9d40-68f0b7c2657d admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:06.316408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-652bb6c8-b4cc-4520-9d40-68f0b7c2657d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d57a50bd11642e19286b07af246109a', 'request_id': 'req-652bb6c8-b4cc-4520-9d40-68f0b7c2657d'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:06.326094 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-652bb6c8-b4cc-4520-9d40-68f0b7c2657d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d57a50bd11642e19286b07af246109a', 'request_id': 'req-652bb6c8-b4cc-4520-9d40-68f0b7c2657d'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:06.332374 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-652bb6c8-b4cc-4520-9d40-68f0b7c2657d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d57a50bd11642e19286b07af246109a', 'request_id': 'req-652bb6c8-b4cc-4520-9d40-68f0b7c2657d'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:06.340973 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:06.345924 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-652bb6c8-b4cc-4520-9d40-68f0b7c2657d admin admin] The token cache is being invalidate because project 2d57a50bd11642e19286b07af246109a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:38:06.346745 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-652bb6c8-b4cc-4520-9d40-68f0b7c2657d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:06.347898 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-652bb6c8-b4cc-4520-9d40-68f0b7c2657d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:06.349543 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1103/2188] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:38:06 2022] DELETE /identity/v3/projects/2d57a50bd11642e19286b07af246109a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:38:06.353772 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b271f2-b48c-43f9-acff-d12e7850af85 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:06.363727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b271f2-b48c-43f9-acff-d12e7850af85 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:06.427400 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643a6d1-2e91-4349-a039-4b2755983997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:06.428058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643a6d1-2e91-4349-a039-4b2755983997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:06.428345 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643a6d1-2e91-4349-a039-4b2755983997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:06.428625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643a6d1-2e91-4349-a039-4b2755983997 admin admin] PATH_INFO: `/v3/users/47a150d586ed42e68d6a5d703b8805e1` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:06.429856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643a6d1-2e91-4349-a039-4b2755983997 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47a150d586ed42e68d6a5d703b8805e1)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:06.431789 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643a6d1-2e91-4349-a039-4b2755983997 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:06.487032 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:06.490243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-1643a6d1-2e91-4349-a039-4b2755983997 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47a150d586ed42e68d6a5d703b8805e1'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:06.503504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-1643a6d1-2e91-4349-a039-4b2755983997 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47a150d586ed42e68d6a5d703b8805e1'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:06.514821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-1643a6d1-2e91-4349-a039-4b2755983997 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47a150d586ed42e68d6a5d703b8805e1'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:06.532869 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-1643a6d1-2e91-4349-a039-4b2755983997 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47a150d586ed42e68d6a5d703b8805e1'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:06.533621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-1643a6d1-2e91-4349-a039-4b2755983997 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47a150d586ed42e68d6a5d703b8805e1'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:06.542597 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1086/2189] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:38:06 2022] DELETE /identity/v3/users/47a150d586ed42e68d6a5d703b8805e1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:38:06.849458 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bb6c8-b4cc-4520-9d40-68f0b7c2657d admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:06.850918 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-652bb6c8-b4cc-4520-9d40-68f0b7c2657d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:06.869335 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bb6c8-b4cc-4520-9d40-68f0b7c2657d admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:06.958954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5c024-f7e3-4fa9-977a-62759a5cf6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:06.959606 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5c024-f7e3-4fa9-977a-62759a5cf6bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:06.959903 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5c024-f7e3-4fa9-977a-62759a5cf6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:06.960231 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5c024-f7e3-4fa9-977a-62759a5cf6bc admin admin] PATH_INFO: `/v3/projects/91a8a870d9f14f56bd3d486115622992` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:06.961368 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5c024-f7e3-4fa9-977a-62759a5cf6bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=91a8a870d9f14f56bd3d486115622992)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:06.963282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5c024-f7e3-4fa9-977a-62759a5cf6bc admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:07.047145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-12e5c024-f7e3-4fa9-977a-62759a5cf6bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91a8a870d9f14f56bd3d486115622992', 'request_id': 'req-12e5c024-f7e3-4fa9-977a-62759a5cf6bc'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:07.059141 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-12e5c024-f7e3-4fa9-977a-62759a5cf6bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91a8a870d9f14f56bd3d486115622992', 'request_id': 'req-12e5c024-f7e3-4fa9-977a-62759a5cf6bc'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:07.063555 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-12e5c024-f7e3-4fa9-977a-62759a5cf6bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91a8a870d9f14f56bd3d486115622992', 'request_id': 'req-12e5c024-f7e3-4fa9-977a-62759a5cf6bc'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:07.076457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-12e5c024-f7e3-4fa9-977a-62759a5cf6bc admin admin] The token cache is being invalidate because project 91a8a870d9f14f56bd3d486115622992 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:38:07.076748 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-12e5c024-f7e3-4fa9-977a-62759a5cf6bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:07.077438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-12e5c024-f7e3-4fa9-977a-62759a5cf6bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:07.092781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1104/2190] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:38:06 2022] DELETE /identity/v3/projects/91a8a870d9f14f56bd3d486115622992 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:38:07.093373 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a9f29-e739-4bc2-a14d-fb71e3fc6209 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:07.093373 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a9f29-e739-4bc2-a14d-fb71e3fc6209 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:07.093373 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a9f29-e739-4bc2-a14d-fb71e3fc6209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:07.115832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-7d1a9f29-e739-4bc2-a14d-fb71e3fc6209 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:07.117119 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-7d1a9f29-e739-4bc2-a14d-fb71e3fc6209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:07.206054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1087/2191] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:38:07.207424 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a0cd2-6a28-43d2-8976-da9c36ee6c90 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:07.207712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a0cd2-6a28-43d2-8976-da9c36ee6c90 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:07.207990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a0cd2-6a28-43d2-8976-da9c36ee6c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:07.232736 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-cf8a0cd2-6a28-43d2-8976-da9c36ee6c90 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:07.234103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-cf8a0cd2-6a28-43d2-8976-da9c36ee6c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:07.258674 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1105/2192] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:38:07.262935 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a9f29-e739-4bc2-a14d-fb71e3fc6209 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:07.277457 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a9f29-e739-4bc2-a14d-fb71e3fc6209 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:07.282681 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f0491-0cfd-4902-81c9-5308f64d7d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:07.283589 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f0491-0cfd-4902-81c9-5308f64d7d3e admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:07.283879 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f0491-0cfd-4902-81c9-5308f64d7d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:07.284153 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f0491-0cfd-4902-81c9-5308f64d7d3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:07.284848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f0491-0cfd-4902-81c9-5308f64d7d3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:38:07.285186 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f0491-0cfd-4902-81c9-5308f64d7d3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:07.286804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f0491-0cfd-4902-81c9-5308f64d7d3e admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:07.299287 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1088/2193] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:38:07.303838 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a0cd2-6a28-43d2-8976-da9c36ee6c90 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:07.313863 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a0cd2-6a28-43d2-8976-da9c36ee6c90 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:07.318593 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876da504-0929-45a5-b0e7-2117599d0d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:07.319209 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876da504-0929-45a5-b0e7-2117599d0d75 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:07.319490 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876da504-0929-45a5-b0e7-2117599d0d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:07.319763 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876da504-0929-45a5-b0e7-2117599d0d75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:07.320428 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876da504-0929-45a5-b0e7-2117599d0d75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:07.322395 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876da504-0929-45a5-b0e7-2117599d0d75 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:07.360124 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1106/2194] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:38:07 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:38:07.364468 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f0491-0cfd-4902-81c9-5308f64d7d3e admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:07.377267 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f0491-0cfd-4902-81c9-5308f64d7d3e admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:07.453888 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc41d88-6a5a-412b-873c-dbca4aa1618b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:07.454628 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc41d88-6a5a-412b-873c-dbca4aa1618b admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:07.454907 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc41d88-6a5a-412b-873c-dbca4aa1618b admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:07.455228 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc41d88-6a5a-412b-873c-dbca4aa1618b admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:07.456117 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc41d88-6a5a-412b-873c-dbca4aa1618b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:07.458086 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc41d88-6a5a-412b-873c-dbca4aa1618b admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:07.482568 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1089/2195] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:38:07 2022] POST /identity/v3/users => generated 285 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:38:07.486774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876da504-0929-45a5-b0e7-2117599d0d75 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:07.501329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876da504-0929-45a5-b0e7-2117599d0d75 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:07.508001 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f3462-46aa-4202-834c-50d3f672d251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:07.508931 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f3462-46aa-4202-834c-50d3f672d251 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:07.509331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f3462-46aa-4202-834c-50d3f672d251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:07.509733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f3462-46aa-4202-834c-50d3f672d251 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:07.510560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f3462-46aa-4202-834c-50d3f672d251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:07.512265 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f3462-46aa-4202-834c-50d3f672d251 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:07.523688 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1107/2196] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:38:07.529253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc41d88-6a5a-412b-873c-dbca4aa1618b admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:07.539675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc41d88-6a5a-412b-873c-dbca4aa1618b admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:07.545328 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07884e45-73ca-4af1-b0e9-c9efd3cb5008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:07.545955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07884e45-73ca-4af1-b0e9-c9efd3cb5008 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:07.546349 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07884e45-73ca-4af1-b0e9-c9efd3cb5008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:07.546732 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07884e45-73ca-4af1-b0e9-c9efd3cb5008 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:07.547566 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07884e45-73ca-4af1-b0e9-c9efd3cb5008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:07.549379 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07884e45-73ca-4af1-b0e9-c9efd3cb5008 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:07.562803 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1090/2197] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:38:07.567174 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f3462-46aa-4202-834c-50d3f672d251 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:07.578074 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f3462-46aa-4202-834c-50d3f672d251 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:07.584282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4eb1ce-63a9-4af3-ac52-2d26649bc7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:07.585148 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4eb1ce-63a9-4af3-ac52-2d26649bc7b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:07.585522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4eb1ce-63a9-4af3-ac52-2d26649bc7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:07.585867 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4eb1ce-63a9-4af3-ac52-2d26649bc7b3 admin admin] PATH_INFO: `/v3/projects/fa113a9566d1479ab2e120d4a14eeb3e/users/f5f64b1bde8148c0ac5b824216f55390/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:07.605254 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4eb1ce-63a9-4af3-ac52-2d26649bc7b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa113a9566d1479ab2e120d4a14eeb3e, user_id=f5f64b1bde8148c0ac5b824216f55390, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:07.608659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4eb1ce-63a9-4af3-ac52-2d26649bc7b3 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:07.623298 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1108/2198] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:38:07 2022] PUT /identity/v3/projects/fa113a9566d1479ab2e120d4a14eeb3e/users/f5f64b1bde8148c0ac5b824216f55390/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:38:07.634424 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4ee41-8c1f-4c58-b72b-f27e4cc5e406 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:07.635063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4ee41-8c1f-4c58-b72b-f27e4cc5e406 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:07.635439 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4ee41-8c1f-4c58-b72b-f27e4cc5e406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:07.669556 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-7fe4ee41-8c1f-4c58-b72b-f27e4cc5e406 None None] MFA Rules not processed for user `f5f64b1bde8148c0ac5b824216f55390`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:07.671180 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-7fe4ee41-8c1f-4c58-b72b-f27e4cc5e406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:07.693032 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:07.750730 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1091/2199] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:38:07 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:38:07.762501 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4eb1ce-63a9-4af3-ac52-2d26649bc7b3 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:07.764344 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-cf4eb1ce-63a9-4af3-ac52-2d26649bc7b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:07.799913 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:07.831640 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4eb1ce-63a9-4af3-ac52-2d26649bc7b3 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:07.835639 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04dd836-2ece-42fc-af32-f9696e12c578 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c50c2d1781b04a5fa8be81bce9205747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:07.836347 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04dd836-2ece-42fc-af32-f9696e12c578 service glance] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:07.836701 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04dd836-2ece-42fc-af32-f9696e12c578 service glance] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:07.837086 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04dd836-2ece-42fc-af32-f9696e12c578 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:07.847195 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04dd836-2ece-42fc-af32-f9696e12c578 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:07.848745 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04dd836-2ece-42fc-af32-f9696e12c578 service glance] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:07.858714 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1109/2200] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:38:07 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:38:17.746721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=19 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:17.854587 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=14 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:20.630724 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9b413-7833-45e7-95d9-508665214a58 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:20.631227 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9b413-7833-45e7-95d9-508665214a58 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:20.631381 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9b413-7833-45e7-95d9-508665214a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:20.653288 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-72d9b413-7833-45e7-95d9-508665214a58 None None] MFA Rules not processed for user `140059c5b30c426ab6d7f1236eb5f684`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:20.654752 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-72d9b413-7833-45e7-95d9-508665214a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:20.725052 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1092/2201] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:38:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:38:23.353452 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04dd836-2ece-42fc-af32-f9696e12c578 service glance] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:23.355074 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-c04dd836-2ece-42fc-af32-f9696e12c578 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:23.440235 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04dd836-2ece-42fc-af32-f9696e12c578 service glance] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:23.445300 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc81d75-694d-4a32-a0fe-5f113401a73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:23.445990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc81d75-694d-4a32-a0fe-5f113401a73b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:23.446275 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc81d75-694d-4a32-a0fe-5f113401a73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:23.446594 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc81d75-694d-4a32-a0fe-5f113401a73b admin admin] PATH_INFO: `/v3/users/06819823875146a99a05cc9b0bfe67ce` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:23.447755 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc81d75-694d-4a32-a0fe-5f113401a73b admin admin] RBAC: Authorizing `identity:delete_user(user_id=06819823875146a99a05cc9b0bfe67ce)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:23.449810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc81d75-694d-4a32-a0fe-5f113401a73b admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:23.506543 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-7bc81d75-694d-4a32-a0fe-5f113401a73b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06819823875146a99a05cc9b0bfe67ce'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:23.517176 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-7bc81d75-694d-4a32-a0fe-5f113401a73b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06819823875146a99a05cc9b0bfe67ce'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:23.530378 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-7bc81d75-694d-4a32-a0fe-5f113401a73b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06819823875146a99a05cc9b0bfe67ce'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:23.537689 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-7bc81d75-694d-4a32-a0fe-5f113401a73b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06819823875146a99a05cc9b0bfe67ce'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:23.538397 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-7bc81d75-694d-4a32-a0fe-5f113401a73b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06819823875146a99a05cc9b0bfe67ce'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:23.545300 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1110/2202] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:38:23 2022] DELETE /identity/v3/users/06819823875146a99a05cc9b0bfe67ce => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:38:23.550010 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1078d2-260f-4aae-969f-1ccbe9398f9f None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:23.550588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1078d2-260f-4aae-969f-1ccbe9398f9f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:23.550588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1078d2-260f-4aae-969f-1ccbe9398f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:23.570964 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-2c1078d2-260f-4aae-969f-1ccbe9398f9f None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:23.572267 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-2c1078d2-260f-4aae-969f-1ccbe9398f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:23.636617 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1093/2203] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:38:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:38:23.646273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc81d75-694d-4a32-a0fe-5f113401a73b admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:23.647940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7bc81d75-694d-4a32-a0fe-5f113401a73b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:23.718504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc81d75-694d-4a32-a0fe-5f113401a73b admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:23.722385 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7f836-3058-415c-a40c-bbf1d1ae1740 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:23.723176 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7f836-3058-415c-a40c-bbf1d1ae1740 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:23.723456 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7f836-3058-415c-a40c-bbf1d1ae1740 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:23.723729 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7f836-3058-415c-a40c-bbf1d1ae1740 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:23.735153 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7f836-3058-415c-a40c-bbf1d1ae1740 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:23.737169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7f836-3058-415c-a40c-bbf1d1ae1740 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:23.749756 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1111/2204] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:38:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:38:24.126230 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1078d2-260f-4aae-969f-1ccbe9398f9f None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:24.128118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-2c1078d2-260f-4aae-969f-1ccbe9398f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:24.188655 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1078d2-260f-4aae-969f-1ccbe9398f9f None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:24.196996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb8963-595a-4cb4-b765-645fe3bc8409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:24.197896 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb8963-595a-4cb4-b765-645fe3bc8409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:24.198289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb8963-595a-4cb4-b765-645fe3bc8409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:24.198650 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb8963-595a-4cb4-b765-645fe3bc8409 admin admin] PATH_INFO: `/v3/projects/f95c1b7268174fce821c7d056058dcde` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:24.201116 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb8963-595a-4cb4-b765-645fe3bc8409 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f95c1b7268174fce821c7d056058dcde)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:24.203685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb8963-595a-4cb4-b765-645fe3bc8409 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:24.259170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-81fb8963-595a-4cb4-b765-645fe3bc8409 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f95c1b7268174fce821c7d056058dcde', 'request_id': 'req-81fb8963-595a-4cb4-b765-645fe3bc8409'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:24.266684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-81fb8963-595a-4cb4-b765-645fe3bc8409 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f95c1b7268174fce821c7d056058dcde', 'request_id': 'req-81fb8963-595a-4cb4-b765-645fe3bc8409'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:24.286462 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-81fb8963-595a-4cb4-b765-645fe3bc8409 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f95c1b7268174fce821c7d056058dcde', 'request_id': 'req-81fb8963-595a-4cb4-b765-645fe3bc8409'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:24.300132 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-81fb8963-595a-4cb4-b765-645fe3bc8409 admin admin] The token cache is being invalidate because project f95c1b7268174fce821c7d056058dcde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:38:24.300432 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-81fb8963-595a-4cb4-b765-645fe3bc8409 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:24.301190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-81fb8963-595a-4cb4-b765-645fe3bc8409 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:24.303145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1094/2205] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:38:24 2022] DELETE /identity/v3/projects/f95c1b7268174fce821c7d056058dcde => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:38:26.600083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d68f7-2e0f-4816-bcd5-a3bec7d13485 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:26.600584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d68f7-2e0f-4816-bcd5-a3bec7d13485 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:26.600673 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d68f7-2e0f-4816-bcd5-a3bec7d13485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:26.621202 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-605d68f7-2e0f-4816-bcd5-a3bec7d13485 None None] MFA Rules not processed for user `07d6bffc69674103b4c6540598a780b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:26.622998 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-605d68f7-2e0f-4816-bcd5-a3bec7d13485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:26.640373 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:26.681245 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1112/2206] 10.222.0.74 () {58 vars in 967 bytes} [Wed Jul 27 16:38:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:38:26.694096 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb8963-595a-4cb4-b765-645fe3bc8409 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:26.695547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-81fb8963-595a-4cb4-b765-645fe3bc8409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:26.729062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:26.769127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb8963-595a-4cb4-b765-645fe3bc8409 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:26.774158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca0478-0a33-4ec6-a540-16fb16028a53 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb45ded9f75e4fa2973b564f83bdfaf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:26.774783 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca0478-0a33-4ec6-a540-16fb16028a53 service swift] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:26.775062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca0478-0a33-4ec6-a540-16fb16028a53 service swift] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:26.775335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca0478-0a33-4ec6-a540-16fb16028a53 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:26.784198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca0478-0a33-4ec6-a540-16fb16028a53 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:26.785847 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca0478-0a33-4ec6-a540-16fb16028a53 service swift] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:26.794299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1095/2207] 10.222.0.74 () {60 vars in 1384 bytes} [Wed Jul 27 16:38:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 16:38:29.249555 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d68f7-2e0f-4816-bcd5-a3bec7d13485 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:29.251178 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-605d68f7-2e0f-4816-bcd5-a3bec7d13485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:29.372705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d68f7-2e0f-4816-bcd5-a3bec7d13485 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:29.377530 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8432d41-b81e-4a03-ad31-36c1426831d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:29.378317 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8432d41-b81e-4a03-ad31-36c1426831d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:29.378609 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8432d41-b81e-4a03-ad31-36c1426831d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:29.378954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8432d41-b81e-4a03-ad31-36c1426831d8 admin admin] PATH_INFO: `/v3/users/4fe9ea35543546afab739c0cfa3389fc` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:29.380259 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8432d41-b81e-4a03-ad31-36c1426831d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fe9ea35543546afab739c0cfa3389fc)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:29.382291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8432d41-b81e-4a03-ad31-36c1426831d8 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:29.474590 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-c8432d41-b81e-4a03-ad31-36c1426831d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fe9ea35543546afab739c0cfa3389fc'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:29.486533 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-c8432d41-b81e-4a03-ad31-36c1426831d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fe9ea35543546afab739c0cfa3389fc'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:29.539251 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-c8432d41-b81e-4a03-ad31-36c1426831d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fe9ea35543546afab739c0cfa3389fc'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:29.561056 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-c8432d41-b81e-4a03-ad31-36c1426831d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fe9ea35543546afab739c0cfa3389fc'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:29.562049 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-c8432d41-b81e-4a03-ad31-36c1426831d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fe9ea35543546afab739c0cfa3389fc'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:29.580278 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1113/2208] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:38:29 2022] DELETE /identity/v3/users/4fe9ea35543546afab739c0cfa3389fc => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:38:30.064441 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca0478-0a33-4ec6-a540-16fb16028a53 service swift] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:30.076844 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca0478-0a33-4ec6-a540-16fb16028a53 service swift] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:30.154134 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1073f-9716-4fae-ac61-6e462789d33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:30.155375 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1073f-9716-4fae-ac61-6e462789d33f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:30.155669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1073f-9716-4fae-ac61-6e462789d33f admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:30.156016 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1073f-9716-4fae-ac61-6e462789d33f admin admin] PATH_INFO: `/v3/projects/fef3e0b606114adfbb6ff0c7e5648bea` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:30.157264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1073f-9716-4fae-ac61-6e462789d33f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fef3e0b606114adfbb6ff0c7e5648bea)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:30.160946 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1073f-9716-4fae-ac61-6e462789d33f admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:30.228842 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-cdc1073f-9716-4fae-ac61-6e462789d33f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fef3e0b606114adfbb6ff0c7e5648bea', 'request_id': 'req-cdc1073f-9716-4fae-ac61-6e462789d33f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:30.240430 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-cdc1073f-9716-4fae-ac61-6e462789d33f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fef3e0b606114adfbb6ff0c7e5648bea', 'request_id': 'req-cdc1073f-9716-4fae-ac61-6e462789d33f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:30.251425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-cdc1073f-9716-4fae-ac61-6e462789d33f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fef3e0b606114adfbb6ff0c7e5648bea', 'request_id': 'req-cdc1073f-9716-4fae-ac61-6e462789d33f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:30.271108 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-cdc1073f-9716-4fae-ac61-6e462789d33f admin admin] The token cache is being invalidate because project fef3e0b606114adfbb6ff0c7e5648bea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:38:30.271450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-cdc1073f-9716-4fae-ac61-6e462789d33f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:30.272354 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-cdc1073f-9716-4fae-ac61-6e462789d33f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:30.275102 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1096/2209] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:38:30 2022] DELETE /identity/v3/projects/fef3e0b606114adfbb6ff0c7e5648bea => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:38:30.280572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079ffff-2dbe-4afc-a3d3-923dbe096768 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:30.281084 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079ffff-2dbe-4afc-a3d3-923dbe096768 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:30.281164 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079ffff-2dbe-4afc-a3d3-923dbe096768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:30.306852 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-d079ffff-2dbe-4afc-a3d3-923dbe096768 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:30.309074 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d079ffff-2dbe-4afc-a3d3-923dbe096768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:30.376004 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:30.403684 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1114/2210] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:38:30.411088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e2446-c501-4b2b-9a1b-62a9097dd721 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:30.411291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e2446-c501-4b2b-9a1b-62a9097dd721 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:30.411568 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e2446-c501-4b2b-9a1b-62a9097dd721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:30.436316 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-025e2446-c501-4b2b-9a1b-62a9097dd721 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:30.436965 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-025e2446-c501-4b2b-9a1b-62a9097dd721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:30.449425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1097/2211] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:38:30.450213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079ffff-2dbe-4afc-a3d3-923dbe096768 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:30.462701 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079ffff-2dbe-4afc-a3d3-923dbe096768 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:30.470980 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a812f8-476f-4c9f-bb3c-8c36268dd029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:30.472021 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a812f8-476f-4c9f-bb3c-8c36268dd029 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:30.472296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a812f8-476f-4c9f-bb3c-8c36268dd029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:30.472567 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a812f8-476f-4c9f-bb3c-8c36268dd029 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:30.473343 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a812f8-476f-4c9f-bb3c-8c36268dd029 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:38:30.473629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a812f8-476f-4c9f-bb3c-8c36268dd029 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:30.477576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a812f8-476f-4c9f-bb3c-8c36268dd029 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:30.491971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1115/2212] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:38:30.497458 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e2446-c501-4b2b-9a1b-62a9097dd721 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:30.511185 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e2446-c501-4b2b-9a1b-62a9097dd721 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:30.521601 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eaded5-1a89-47cf-9df1-55c919a54475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:30.523661 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eaded5-1a89-47cf-9df1-55c919a54475 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:30.524207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eaded5-1a89-47cf-9df1-55c919a54475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:30.524763 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eaded5-1a89-47cf-9df1-55c919a54475 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:30.526096 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eaded5-1a89-47cf-9df1-55c919a54475 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:30.530409 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eaded5-1a89-47cf-9df1-55c919a54475 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:30.576785 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1098/2213] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:38:30 2022] POST /identity/v3/projects => generated 371 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:38:30.580862 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a812f8-476f-4c9f-bb3c-8c36268dd029 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:30.593616 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a812f8-476f-4c9f-bb3c-8c36268dd029 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:30.658999 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f241dab-2bd0-44e7-8f3a-5e68cc73f812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:30.659742 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f241dab-2bd0-44e7-8f3a-5e68cc73f812 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:30.660091 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f241dab-2bd0-44e7-8f3a-5e68cc73f812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:30.660429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f241dab-2bd0-44e7-8f3a-5e68cc73f812 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:30.661268 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f241dab-2bd0-44e7-8f3a-5e68cc73f812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:30.663479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f241dab-2bd0-44e7-8f3a-5e68cc73f812 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:30.686250 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1116/2214] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:38:30 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:38:30.690451 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eaded5-1a89-47cf-9df1-55c919a54475 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:30.702312 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eaded5-1a89-47cf-9df1-55c919a54475 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:30.707250 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15726c9-c0b0-4e2c-bea0-4dbcbd4469cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:30.708036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15726c9-c0b0-4e2c-bea0-4dbcbd4469cd admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:30.708371 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15726c9-c0b0-4e2c-bea0-4dbcbd4469cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:30.708811 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15726c9-c0b0-4e2c-bea0-4dbcbd4469cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:30.709638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15726c9-c0b0-4e2c-bea0-4dbcbd4469cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:30.711643 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15726c9-c0b0-4e2c-bea0-4dbcbd4469cd admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:30.728518 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1099/2215] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:38:30.733996 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f241dab-2bd0-44e7-8f3a-5e68cc73f812 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:30.745142 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f241dab-2bd0-44e7-8f3a-5e68cc73f812 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:30.752181 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce7c1f-1b56-4c3c-af46-862f47354792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:30.753323 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce7c1f-1b56-4c3c-af46-862f47354792 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:30.753839 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce7c1f-1b56-4c3c-af46-862f47354792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:30.754312 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce7c1f-1b56-4c3c-af46-862f47354792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:30.755086 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce7c1f-1b56-4c3c-af46-862f47354792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:30.757514 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce7c1f-1b56-4c3c-af46-862f47354792 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:30.775704 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1117/2216] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:38:30.778150 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15726c9-c0b0-4e2c-bea0-4dbcbd4469cd admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:30.789227 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15726c9-c0b0-4e2c-bea0-4dbcbd4469cd admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:30.794562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3739d12-6e89-45a8-8121-07aedaed92f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:30.795329 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3739d12-6e89-45a8-8121-07aedaed92f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:30.795642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3739d12-6e89-45a8-8121-07aedaed92f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:30.795910 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3739d12-6e89-45a8-8121-07aedaed92f9 admin admin] PATH_INFO: `/v3/projects/d2acc328fa61402785678a45d606b117/users/c9ed16c7afc941328fcb0362025b777b/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:30.800428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:30.815043 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3739d12-6e89-45a8-8121-07aedaed92f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2acc328fa61402785678a45d606b117, user_id=c9ed16c7afc941328fcb0362025b777b, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:30.818576 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3739d12-6e89-45a8-8121-07aedaed92f9 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:30.835804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1100/2217] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:38:30 2022] PUT /identity/v3/projects/d2acc328fa61402785678a45d606b117/users/c9ed16c7afc941328fcb0362025b777b/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:38:30.848271 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f08ce-eb8b-4e04-bb8b-ce0ff53a5700 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:30.848557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f08ce-eb8b-4e04-bb8b-ce0ff53a5700 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:30.848831 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f08ce-eb8b-4e04-bb8b-ce0ff53a5700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:30.876923 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-382f08ce-eb8b-4e04-bb8b-ce0ff53a5700 None None] MFA Rules not processed for user `c9ed16c7afc941328fcb0362025b777b`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:30.878143 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-382f08ce-eb8b-4e04-bb8b-ce0ff53a5700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:30.941399 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1118/2218] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:38:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:38:30.953669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3739d12-6e89-45a8-8121-07aedaed92f9 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:30.955385 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-a3739d12-6e89-45a8-8121-07aedaed92f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:31.012670 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3739d12-6e89-45a8-8121-07aedaed92f9 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:31.016361 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81bd61-462d-4302-be07-9a528b14836c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:31.017011 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81bd61-462d-4302-be07-9a528b14836c service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:31.017292 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81bd61-462d-4302-be07-9a528b14836c service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:31.017563 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81bd61-462d-4302-be07-9a528b14836c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:31.026172 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81bd61-462d-4302-be07-9a528b14836c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:31.027624 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81bd61-462d-4302-be07-9a528b14836c service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:31.036625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1101/2219] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:38:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:38:31.061737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f08ce-eb8b-4e04-bb8b-ce0ff53a5700 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:31.115001 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:31.135825 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f08ce-eb8b-4e04-bb8b-ce0ff53a5700 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:31.141169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c4ca0-871b-4549-91da-a39111396a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:31.141963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c4ca0-871b-4549-91da-a39111396a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:31.142253 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c4ca0-871b-4549-91da-a39111396a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:31.142522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c4ca0-871b-4549-91da-a39111396a3c admin admin] PATH_INFO: `/v3/users/c9ed16c7afc941328fcb0362025b777b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:31.143691 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c4ca0-871b-4549-91da-a39111396a3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9ed16c7afc941328fcb0362025b777b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:31.145679 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c4ca0-871b-4549-91da-a39111396a3c admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:31.201264 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-065c4ca0-871b-4549-91da-a39111396a3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9ed16c7afc941328fcb0362025b777b'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:31.214315 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-065c4ca0-871b-4549-91da-a39111396a3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9ed16c7afc941328fcb0362025b777b'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:31.229509 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-065c4ca0-871b-4549-91da-a39111396a3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9ed16c7afc941328fcb0362025b777b'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:31.239372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-065c4ca0-871b-4549-91da-a39111396a3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9ed16c7afc941328fcb0362025b777b'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:31.240133 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-065c4ca0-871b-4549-91da-a39111396a3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9ed16c7afc941328fcb0362025b777b'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:31.250507 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1119/2220] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:38:31 2022] DELETE /identity/v3/users/c9ed16c7afc941328fcb0362025b777b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:38:31.251904 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58ced2-e4f8-40c9-a4b8-dc30b31e20fa None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:31.252346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58ced2-e4f8-40c9-a4b8-dc30b31e20fa None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:31.252592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58ced2-e4f8-40c9-a4b8-dc30b31e20fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:31.272703 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-6d58ced2-e4f8-40c9-a4b8-dc30b31e20fa None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:31.273952 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6d58ced2-e4f8-40c9-a4b8-dc30b31e20fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:31.342975 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1102/2221] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:38:31.352620 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c4ca0-871b-4549-91da-a39111396a3c admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:31.354103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-065c4ca0-871b-4549-91da-a39111396a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:31.412121 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c4ca0-871b-4549-91da-a39111396a3c admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:31.416273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f18635-1038-4272-8b0c-0f04685525d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:31.417003 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f18635-1038-4272-8b0c-0f04685525d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:31.417310 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f18635-1038-4272-8b0c-0f04685525d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:31.417592 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f18635-1038-4272-8b0c-0f04685525d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:31.427725 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f18635-1038-4272-8b0c-0f04685525d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:31.429360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f18635-1038-4272-8b0c-0f04685525d4 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:31.440185 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1120/2222] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:38:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:38:31.707287 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58ced2-e4f8-40c9-a4b8-dc30b31e20fa None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:31.708754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6d58ced2-e4f8-40c9-a4b8-dc30b31e20fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:31.726184 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58ced2-e4f8-40c9-a4b8-dc30b31e20fa None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:31.731517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a7c2e-3483-42a8-a235-f9316325abbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:31.732250 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a7c2e-3483-42a8-a235-f9316325abbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:31.732605 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a7c2e-3483-42a8-a235-f9316325abbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:31.732789 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a7c2e-3483-42a8-a235-f9316325abbb admin admin] PATH_INFO: `/v3/projects/d2acc328fa61402785678a45d606b117` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:31.734047 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4a7c2e-3483-42a8-a235-f9316325abbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2acc328fa61402785678a45d606b117)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:31.736123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4a7c2e-3483-42a8-a235-f9316325abbb admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:31.783226 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-bf4a7c2e-3483-42a8-a235-f9316325abbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2acc328fa61402785678a45d606b117', 'request_id': 'req-bf4a7c2e-3483-42a8-a235-f9316325abbb'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:31.788853 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-bf4a7c2e-3483-42a8-a235-f9316325abbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2acc328fa61402785678a45d606b117', 'request_id': 'req-bf4a7c2e-3483-42a8-a235-f9316325abbb'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:31.796459 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-bf4a7c2e-3483-42a8-a235-f9316325abbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2acc328fa61402785678a45d606b117', 'request_id': 'req-bf4a7c2e-3483-42a8-a235-f9316325abbb'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:31.807538 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-bf4a7c2e-3483-42a8-a235-f9316325abbb admin admin] The token cache is being invalidate because project d2acc328fa61402785678a45d606b117 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:38:31.807952 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-bf4a7c2e-3483-42a8-a235-f9316325abbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:31.808574 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-bf4a7c2e-3483-42a8-a235-f9316325abbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:31.810159 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1103/2223] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:38:31 2022] DELETE /identity/v3/projects/d2acc328fa61402785678a45d606b117 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:38:31.815308 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c69b2d-7d2b-418d-83d4-f8f7d8cedccc None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:31.815624 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c69b2d-7d2b-418d-83d4-f8f7d8cedccc None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:31.815930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c69b2d-7d2b-418d-83d4-f8f7d8cedccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:31.836864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-08c69b2d-7d2b-418d-83d4-f8f7d8cedccc None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:31.838096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-08c69b2d-7d2b-418d-83d4-f8f7d8cedccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:31.897504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1121/2224] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:38:31.905610 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171a1cf-0321-4061-aebe-0034db08364c None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:31.905899 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171a1cf-0321-4061-aebe-0034db08364c None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:31.906173 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171a1cf-0321-4061-aebe-0034db08364c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:31.925784 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-1171a1cf-0321-4061-aebe-0034db08364c None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:31.927013 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-1171a1cf-0321-4061-aebe-0034db08364c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:31.934785 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1104/2225] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:38:31.938821 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c69b2d-7d2b-418d-83d4-f8f7d8cedccc None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:31.944738 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:31.948554 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c69b2d-7d2b-418d-83d4-f8f7d8cedccc None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:31.954759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da3a9e-825e-4876-886f-8dc4e323dd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:31.955560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da3a9e-825e-4876-886f-8dc4e323dd54 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:31.956041 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da3a9e-825e-4876-886f-8dc4e323dd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:31.956357 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da3a9e-825e-4876-886f-8dc4e323dd54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:31.957230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da3a9e-825e-4876-886f-8dc4e323dd54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:38:31.957583 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da3a9e-825e-4876-886f-8dc4e323dd54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:31.959162 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da3a9e-825e-4876-886f-8dc4e323dd54 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:31.971042 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1122/2226] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:38:31.975104 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171a1cf-0321-4061-aebe-0034db08364c None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:31.981265 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:31.985741 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171a1cf-0321-4061-aebe-0034db08364c None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:31.993152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fecdfc-e343-4e16-9eb3-e257edff7b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:31.994227 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fecdfc-e343-4e16-9eb3-e257edff7b93 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:31.994588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fecdfc-e343-4e16-9eb3-e257edff7b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:31.994919 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fecdfc-e343-4e16-9eb3-e257edff7b93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:31.996158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fecdfc-e343-4e16-9eb3-e257edff7b93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:31.998715 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fecdfc-e343-4e16-9eb3-e257edff7b93 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:32.028849 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1105/2227] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:38:31 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:38:32.030697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da3a9e-825e-4876-886f-8dc4e323dd54 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:32.041284 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da3a9e-825e-4876-886f-8dc4e323dd54 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:32.101393 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b60322-8ecb-4e50-953d-6655d1a97f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:32.102068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b60322-8ecb-4e50-953d-6655d1a97f4a admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:32.102393 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b60322-8ecb-4e50-953d-6655d1a97f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:32.102675 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b60322-8ecb-4e50-953d-6655d1a97f4a admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:32.103342 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b60322-8ecb-4e50-953d-6655d1a97f4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:32.105418 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b60322-8ecb-4e50-953d-6655d1a97f4a admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:32.126317 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1123/2228] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:38:32 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:38:32.130595 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fecdfc-e343-4e16-9eb3-e257edff7b93 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:32.141478 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fecdfc-e343-4e16-9eb3-e257edff7b93 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:32.146064 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35de5ed-48fb-45dd-8926-2b56a3f30aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:32.146670 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35de5ed-48fb-45dd-8926-2b56a3f30aae admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:32.146952 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35de5ed-48fb-45dd-8926-2b56a3f30aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:32.147225 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35de5ed-48fb-45dd-8926-2b56a3f30aae admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:32.147906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35de5ed-48fb-45dd-8926-2b56a3f30aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:32.149662 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35de5ed-48fb-45dd-8926-2b56a3f30aae admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:32.160597 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1106/2229] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:38:32.164564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b60322-8ecb-4e50-953d-6655d1a97f4a admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:32.174661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b60322-8ecb-4e50-953d-6655d1a97f4a admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:32.179371 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6423c98-8709-4763-b37c-7d87731ef594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:32.179981 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6423c98-8709-4763-b37c-7d87731ef594 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:32.180263 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6423c98-8709-4763-b37c-7d87731ef594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:32.180536 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6423c98-8709-4763-b37c-7d87731ef594 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:32.181259 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6423c98-8709-4763-b37c-7d87731ef594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:32.182855 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6423c98-8709-4763-b37c-7d87731ef594 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:32.193263 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1124/2230] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:38:32.197520 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35de5ed-48fb-45dd-8926-2b56a3f30aae admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:32.207865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35de5ed-48fb-45dd-8926-2b56a3f30aae admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:32.212440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84589f5-6547-443d-a7dc-9cd100d2a8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:32.213245 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84589f5-6547-443d-a7dc-9cd100d2a8af admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:32.213618 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84589f5-6547-443d-a7dc-9cd100d2a8af admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:32.213897 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84589f5-6547-443d-a7dc-9cd100d2a8af admin admin] PATH_INFO: `/v3/projects/ce1f193c68ef49b78c95c02f7413b7d6/users/00f17de515184c01a73c454c7f0b7e70/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:32.229881 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84589f5-6547-443d-a7dc-9cd100d2a8af admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce1f193c68ef49b78c95c02f7413b7d6, user_id=00f17de515184c01a73c454c7f0b7e70, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:32.232550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84589f5-6547-443d-a7dc-9cd100d2a8af admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:32.245964 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1107/2231] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:38:32 2022] PUT /identity/v3/projects/ce1f193c68ef49b78c95c02f7413b7d6/users/00f17de515184c01a73c454c7f0b7e70/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:38:32.250953 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424dc43-3aae-47a4-b10f-8baac9390b04 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:32.251250 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424dc43-3aae-47a4-b10f-8baac9390b04 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:32.251547 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424dc43-3aae-47a4-b10f-8baac9390b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:32.270408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-e424dc43-3aae-47a4-b10f-8baac9390b04 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:32.271610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e424dc43-3aae-47a4-b10f-8baac9390b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:32.333750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1125/2232] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:38:32.344365 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84589f5-6547-443d-a7dc-9cd100d2a8af admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:32.346288 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-a84589f5-6547-443d-a7dc-9cd100d2a8af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:32.406347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84589f5-6547-443d-a7dc-9cd100d2a8af admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:32.410526 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda198fa-a654-4af6-ab82-aef8a552df21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:32.411177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda198fa-a654-4af6-ab82-aef8a552df21 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:32.411461 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda198fa-a654-4af6-ab82-aef8a552df21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:32.411761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda198fa-a654-4af6-ab82-aef8a552df21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:32.421631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda198fa-a654-4af6-ab82-aef8a552df21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:32.423241 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda198fa-a654-4af6-ab82-aef8a552df21 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:32.433883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1108/2233] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:38:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:38:33.267111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bcacd-fe74-49b0-bdc5-9118f7f98454 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:33.267841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bcacd-fe74-49b0-bdc5-9118f7f98454 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:33.267841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bcacd-fe74-49b0-bdc5-9118f7f98454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:33.300028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-1a4bcacd-fe74-49b0-bdc5-9118f7f98454 None None] MFA Rules not processed for user `00f17de515184c01a73c454c7f0b7e70`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:33.302276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-1a4bcacd-fe74-49b0-bdc5-9118f7f98454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:33.368510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:33.373943 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1126/2234] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:38:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:38:33.384970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda198fa-a654-4af6-ab82-aef8a552df21 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:33.386518 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-eda198fa-a654-4af6-ab82-aef8a552df21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:33.446823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda198fa-a654-4af6-ab82-aef8a552df21 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:33.450606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac397f0-b739-4737-9916-ad237cac78d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:33.452450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac397f0-b739-4737-9916-ad237cac78d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:33.452766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac397f0-b739-4737-9916-ad237cac78d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:33.453084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac397f0-b739-4737-9916-ad237cac78d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:33.462354 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac397f0-b739-4737-9916-ad237cac78d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:33.463929 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac397f0-b739-4737-9916-ad237cac78d4 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:33.474046 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1109/2235] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:38:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:38:34.866133 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a3499-e0f9-4880-bc69-c0ec4b7a4e4c None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:34.866644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a3499-e0f9-4880-bc69-c0ec4b7a4e4c None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:34.866740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a3499-e0f9-4880-bc69-c0ec4b7a4e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:34.885733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-0f8a3499-e0f9-4880-bc69-c0ec4b7a4e4c None None] MFA Rules not processed for user `00f17de515184c01a73c454c7f0b7e70`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:34.886959 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-0f8a3499-e0f9-4880-bc69-c0ec4b7a4e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:34.895893 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1127/2236] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:38:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:38:39.264846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7020ad-4423-4405-90ba-a6132e5cd226 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:39.265418 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7020ad-4423-4405-90ba-a6132e5cd226 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:39.265704 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7020ad-4423-4405-90ba-a6132e5cd226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:39.284651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-ac7020ad-4423-4405-90ba-a6132e5cd226 None None] MFA Rules not processed for user `1c0838069cba4d1389ec6fc809db70a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:39.285893 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ac7020ad-4423-4405-90ba-a6132e5cd226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:39.333307 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:39.343796 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1110/2237] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:38:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:38:44.880125 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:45.804949 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a3499-e0f9-4880-bc69-c0ec4b7a4e4c None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:45.806332 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-0f8a3499-e0f9-4880-bc69-c0ec4b7a4e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:45.820351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a3499-e0f9-4880-bc69-c0ec4b7a4e4c None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:45.824802 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f034c73-3f13-4c93-91ab-9f860de5b20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:45.825470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f034c73-3f13-4c93-91ab-9f860de5b20d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:45.825753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f034c73-3f13-4c93-91ab-9f860de5b20d admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:45.826028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f034c73-3f13-4c93-91ab-9f860de5b20d admin admin] PATH_INFO: `/v3/users/f5f64b1bde8148c0ac5b824216f55390` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:45.827168 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f034c73-3f13-4c93-91ab-9f860de5b20d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5f64b1bde8148c0ac5b824216f55390)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:45.829114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f034c73-3f13-4c93-91ab-9f860de5b20d admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:45.875492 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6f034c73-3f13-4c93-91ab-9f860de5b20d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5f64b1bde8148c0ac5b824216f55390'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:45.882883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6f034c73-3f13-4c93-91ab-9f860de5b20d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5f64b1bde8148c0ac5b824216f55390'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:45.894900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6f034c73-3f13-4c93-91ab-9f860de5b20d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5f64b1bde8148c0ac5b824216f55390'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:45.901684 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6f034c73-3f13-4c93-91ab-9f860de5b20d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5f64b1bde8148c0ac5b824216f55390'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:45.902382 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6f034c73-3f13-4c93-91ab-9f860de5b20d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5f64b1bde8148c0ac5b824216f55390'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:45.908393 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1128/2238] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:38:45 2022] DELETE /identity/v3/users/f5f64b1bde8148c0ac5b824216f55390 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:38:45.912630 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0f7e3-df74-43b8-b0dd-f552ef26be68 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:45.913134 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0f7e3-df74-43b8-b0dd-f552ef26be68 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:45.913241 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0f7e3-df74-43b8-b0dd-f552ef26be68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:45.932752 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-01a0f7e3-df74-43b8-b0dd-f552ef26be68 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:45.934738 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-01a0f7e3-df74-43b8-b0dd-f552ef26be68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:45.993447 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1111/2239] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:38:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:38:46.004703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f034c73-3f13-4c93-91ab-9f860de5b20d admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:46.006153 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-6f034c73-3f13-4c93-91ab-9f860de5b20d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:46.067487 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f034c73-3f13-4c93-91ab-9f860de5b20d admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:46.071602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d110b-04d6-44cf-a192-5a1d835d3d77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:46.072314 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d110b-04d6-44cf-a192-5a1d835d3d77 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:46.072663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d110b-04d6-44cf-a192-5a1d835d3d77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:46.072978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d110b-04d6-44cf-a192-5a1d835d3d77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:46.083202 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6d110b-04d6-44cf-a192-5a1d835d3d77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:46.084865 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6d110b-04d6-44cf-a192-5a1d835d3d77 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:46.096422 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1129/2240] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:38:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:38:46.375583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0f7e3-df74-43b8-b0dd-f552ef26be68 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:46.377249 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-01a0f7e3-df74-43b8-b0dd-f552ef26be68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:46.393669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0f7e3-df74-43b8-b0dd-f552ef26be68 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:46.400336 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb633f8-d350-4c86-95c2-fa41ef3caf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:46.401110 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb633f8-d350-4c86-95c2-fa41ef3caf7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:46.401443 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb633f8-d350-4c86-95c2-fa41ef3caf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:46.401976 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb633f8-d350-4c86-95c2-fa41ef3caf7a admin admin] PATH_INFO: `/v3/projects/fa113a9566d1479ab2e120d4a14eeb3e` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:46.403373 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb633f8-d350-4c86-95c2-fa41ef3caf7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa113a9566d1479ab2e120d4a14eeb3e)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:46.405721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb633f8-d350-4c86-95c2-fa41ef3caf7a admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:46.457651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-3eb633f8-d350-4c86-95c2-fa41ef3caf7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa113a9566d1479ab2e120d4a14eeb3e', 'request_id': 'req-3eb633f8-d350-4c86-95c2-fa41ef3caf7a'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:46.463382 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-3eb633f8-d350-4c86-95c2-fa41ef3caf7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa113a9566d1479ab2e120d4a14eeb3e', 'request_id': 'req-3eb633f8-d350-4c86-95c2-fa41ef3caf7a'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:46.472153 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-3eb633f8-d350-4c86-95c2-fa41ef3caf7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa113a9566d1479ab2e120d4a14eeb3e', 'request_id': 'req-3eb633f8-d350-4c86-95c2-fa41ef3caf7a'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:46.488552 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-3eb633f8-d350-4c86-95c2-fa41ef3caf7a admin admin] The token cache is being invalidate because project fa113a9566d1479ab2e120d4a14eeb3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:38:46.488868 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-3eb633f8-d350-4c86-95c2-fa41ef3caf7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:46.489585 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-3eb633f8-d350-4c86-95c2-fa41ef3caf7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:38:46.491301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1112/2241] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:38:46 2022] DELETE /identity/v3/projects/fa113a9566d1479ab2e120d4a14eeb3e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:38:56.091716 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:56.337076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d110b-04d6-44cf-a192-5a1d835d3d77 service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:56.339728 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-de6d110b-04d6-44cf-a192-5a1d835d3d77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:56.477929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d110b-04d6-44cf-a192-5a1d835d3d77 service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:56.484512 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:38:56.553437 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27561f85-365e-4004-bf4f-96e1d33bc7b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f8e0f82a8541aead6cd8cc45f4a86b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:56.554336 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27561f85-365e-4004-bf4f-96e1d33bc7b6 service placement] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:56.554626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27561f85-365e-4004-bf4f-96e1d33bc7b6 service placement] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:56.554936 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27561f85-365e-4004-bf4f-96e1d33bc7b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:56.557531 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-27561f85-365e-4004-bf4f-96e1d33bc7b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:56.570594 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27561f85-365e-4004-bf4f-96e1d33bc7b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:56.572388 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27561f85-365e-4004-bf4f-96e1d33bc7b6 service placement] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:56.584126 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1130/2242] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:38:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:38:59.437526 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d6989-f215-4a30-9119-44a16579083d None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:59.438058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d6989-f215-4a30-9119-44a16579083d None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:59.438058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d6989-f215-4a30-9119-44a16579083d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:59.458247 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-246d6989-f215-4a30-9119-44a16579083d None None] MFA Rules not processed for user `07d6bffc69674103b4c6540598a780b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:38:59.459496 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-246d6989-f215-4a30-9119-44a16579083d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:59.565439 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1113/2243] 10.222.0.74 () {58 vars in 1009 bytes} [Wed Jul 27 16:38:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:38:59.585545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27561f85-365e-4004-bf4f-96e1d33bc7b6 service placement] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:38:59.587069 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-27561f85-365e-4004-bf4f-96e1d33bc7b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:38:59.659446 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27561f85-365e-4004-bf4f-96e1d33bc7b6 service placement] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:38:59.664479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdcc52b-1d2b-42ea-8269-9b00ea4fad4b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb45ded9f75e4fa2973b564f83bdfaf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:38:59.665135 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcc52b-1d2b-42ea-8269-9b00ea4fad4b service swift] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:38:59.665424 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcc52b-1d2b-42ea-8269-9b00ea4fad4b service swift] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:38:59.665702 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdcc52b-1d2b-42ea-8269-9b00ea4fad4b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:38:59.674832 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdcc52b-1d2b-42ea-8269-9b00ea4fad4b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:38:59.676513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdcc52b-1d2b-42ea-8269-9b00ea4fad4b service swift] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:38:59.685532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1131/2244] 10.222.0.74 () {60 vars in 1384 bytes} [Wed Jul 27 16:38:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 16:39:09.550668 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=25 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:39:09.681576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=90 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:39:13.603830 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ffadc-3c2f-47db-8ee1-e060d12d3c34 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:39:13.604347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ffadc-3c2f-47db-8ee1-e060d12d3c34 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:39:13.604463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ffadc-3c2f-47db-8ee1-e060d12d3c34 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:39:13.605898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1114/2245] 10.222.0.74 () {54 vars in 921 bytes} [Wed Jul 27 16:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:39:13.613533 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa5608-1824-49ec-a722-8270b0c8e5b8 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:39:13.614092 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa5608-1824-49ec-a722-8270b0c8e5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:39:13.614092 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa5608-1824-49ec-a722-8270b0c8e5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:39:13.636066 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-50fa5608-1824-49ec-a722-8270b0c8e5b8 None None] MFA Rules not processed for user `56dfafd15e284b7986ba07dd3819bf65`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:39:13.637328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-50fa5608-1824-49ec-a722-8270b0c8e5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:39:13.710868 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1132/2246] 10.222.0.74 () {58 vars in 1018 bytes} [Wed Jul 27 16:39:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:39:13.734064 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ffadc-3c2f-47db-8ee1-e060d12d3c34 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:39:13.735853 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-eb2ffadc-3c2f-47db-8ee1-e060d12d3c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:39:13.799248 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ffadc-3c2f-47db-8ee1-e060d12d3c34 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:39:13.802816 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05c6a0-a534-40e4-8ff9-b8c4f29f7857 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c50c2d1781b04a5fa8be81bce9205747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:39:13.803432 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05c6a0-a534-40e4-8ff9-b8c4f29f7857 service glance] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:39:13.803717 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05c6a0-a534-40e4-8ff9-b8c4f29f7857 service glance] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:39:13.803997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05c6a0-a534-40e4-8ff9-b8c4f29f7857 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:39:13.814174 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05c6a0-a534-40e4-8ff9-b8c4f29f7857 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:39:13.815631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05c6a0-a534-40e4-8ff9-b8c4f29f7857 service glance] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:39:13.826729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1115/2247] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:39:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:39:13.894483 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c50877-ad3b-4770-b49f-db689450037c None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:39:13.894775 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c50877-ad3b-4770-b49f-db689450037c None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:39:13.895085 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c50877-ad3b-4770-b49f-db689450037c None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:39:13.896637 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1133/2248] 10.222.0.74 () {54 vars in 921 bytes} [Wed Jul 27 16:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:39:13.901943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079770a-e262-4b3d-8f98-d39dd172f83b None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:39:13.902321 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079770a-e262-4b3d-8f98-d39dd172f83b None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:39:13.902693 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079770a-e262-4b3d-8f98-d39dd172f83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:39:13.923109 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-2079770a-e262-4b3d-8f98-d39dd172f83b None None] MFA Rules not processed for user `193dd9fee1844275ad0dab602e93df11`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:39:13.924323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-2079770a-e262-4b3d-8f98-d39dd172f83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:39:13.975713 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1116/2249] 10.222.0.74 () {58 vars in 1018 bytes} [Wed Jul 27 16:39:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:39:13.988223 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c50877-ad3b-4770-b49f-db689450037c None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:39:13.989932 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-03c50877-ad3b-4770-b49f-db689450037c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:39:14.004953 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c50877-ad3b-4770-b49f-db689450037c None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:39:14.008524 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4cb11a-c57c-4213-9f48-607900115a7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:39:14.009155 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cb11a-c57c-4213-9f48-607900115a7c service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:39:14.009432 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cb11a-c57c-4213-9f48-607900115a7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:39:14.009699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cb11a-c57c-4213-9f48-607900115a7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:39:14.016785 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4cb11a-c57c-4213-9f48-607900115a7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:39:14.018249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4cb11a-c57c-4213-9f48-607900115a7c service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:39:14.026952 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1134/2250] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:39:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:39:14.114171 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d8bde-a3c4-4460-8ab4-f4e9d558df4a None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:39:14.114462 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d8bde-a3c4-4460-8ab4-f4e9d558df4a None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:39:14.114749 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d8bde-a3c4-4460-8ab4-f4e9d558df4a None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:39:14.116106 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1117/2251] 10.222.0.74 () {54 vars in 921 bytes} [Wed Jul 27 16:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:39:14.121707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530f52d-baa1-43af-9676-5aa10b3fa7c2 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:39:14.122755 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530f52d-baa1-43af-9676-5aa10b3fa7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:39:14.123984 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530f52d-baa1-43af-9676-5aa10b3fa7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:39:14.143436 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-b530f52d-baa1-43af-9676-5aa10b3fa7c2 None None] MFA Rules not processed for user `56dfafd15e284b7986ba07dd3819bf65`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:39:14.144705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b530f52d-baa1-43af-9676-5aa10b3fa7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:39:14.155293 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1135/2252] 10.222.0.74 () {58 vars in 1018 bytes} [Wed Jul 27 16:39:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:39:14.165896 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d8bde-a3c4-4460-8ab4-f4e9d558df4a None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:39:14.167681 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c65d8bde-a3c4-4460-8ab4-f4e9d558df4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:39:14.181539 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d8bde-a3c4-4460-8ab4-f4e9d558df4a None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:39:14.185710 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a13595-28e5-4bb8-ad8c-4bc4c6c9c44f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:39:14.186318 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a13595-28e5-4bb8-ad8c-4bc4c6c9c44f service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:39:14.186600 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a13595-28e5-4bb8-ad8c-4bc4c6c9c44f service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:39:14.186875 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a13595-28e5-4bb8-ad8c-4bc4c6c9c44f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:39:14.196937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a13595-28e5-4bb8-ad8c-4bc4c6c9c44f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:39:14.198366 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a13595-28e5-4bb8-ad8c-4bc4c6c9c44f service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:39:14.208877 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1118/2253] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:39:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:39:16.883447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198df44a-7f78-41cd-8bb6-178ccf890c5b None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:39:16.883895 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198df44a-7f78-41cd-8bb6-178ccf890c5b None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:39:16.883999 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198df44a-7f78-41cd-8bb6-178ccf890c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:39:16.903754 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-198df44a-7f78-41cd-8bb6-178ccf890c5b None None] MFA Rules not processed for user `07d6bffc69674103b4c6540598a780b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:39:16.905268 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-198df44a-7f78-41cd-8bb6-178ccf890c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:39:16.913099 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1136/2254] 10.222.0.74 () {58 vars in 1009 bytes} [Wed Jul 27 16:39:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:39:16.924940 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a13595-28e5-4bb8-ad8c-4bc4c6c9c44f service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:39:16.937073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a13595-28e5-4bb8-ad8c-4bc4c6c9c44f service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:39:16.942179 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b31deb-7e30-4959-98d7-319d0982a7a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb45ded9f75e4fa2973b564f83bdfaf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:39:16.942804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b31deb-7e30-4959-98d7-319d0982a7a0 service swift] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:39:16.943095 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b31deb-7e30-4959-98d7-319d0982a7a0 service swift] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:39:16.943369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b31deb-7e30-4959-98d7-319d0982a7a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:39:16.952402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b31deb-7e30-4959-98d7-319d0982a7a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:39:16.954012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b31deb-7e30-4959-98d7-319d0982a7a0 service swift] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:39:16.962372 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1119/2255] 10.222.0.74 () {60 vars in 1384 bytes} [Wed Jul 27 16:39:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 16:39:26.898259 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=43 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:39:26.958881 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=61 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:39:46.462778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be924980-6645-4d1c-abc4-f34567699ce1 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:39:46.463324 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be924980-6645-4d1c-abc4-f34567699ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:39:46.463324 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be924980-6645-4d1c-abc4-f34567699ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:39:46.494296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-be924980-6645-4d1c-abc4-f34567699ce1 None None] MFA Rules not processed for user `07d6bffc69674103b4c6540598a780b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:39:46.495913 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-be924980-6645-4d1c-abc4-f34567699ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:39:46.506914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1137/2256] 10.222.0.74 () {58 vars in 967 bytes} [Wed Jul 27 16:39:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:39:46.525915 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b31deb-7e30-4959-98d7-319d0982a7a0 service swift] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:39:46.539232 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b31deb-7e30-4959-98d7-319d0982a7a0 service swift] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:39:46.545654 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2de24-729d-4027-a221-2b891e775aec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb45ded9f75e4fa2973b564f83bdfaf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:39:46.546437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2de24-729d-4027-a221-2b891e775aec service swift] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:39:46.546860 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2de24-729d-4027-a221-2b891e775aec service swift] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:39:46.547806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2de24-729d-4027-a221-2b891e775aec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:39:46.559728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b2de24-729d-4027-a221-2b891e775aec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:39:46.562309 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b2de24-729d-4027-a221-2b891e775aec service swift] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:39:46.573684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1120/2257] 10.222.0.74 () {60 vars in 1384 bytes} [Wed Jul 27 16:39:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 16:39:56.484705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:39:56.568759 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=6 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:39:59.376865 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275362a-9952-4f99-b4b3-f23480abfd85 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:39:59.377401 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275362a-9952-4f99-b4b3-f23480abfd85 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:39:59.377521 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275362a-9952-4f99-b4b3-f23480abfd85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:39:59.396406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-9275362a-9952-4f99-b4b3-f23480abfd85 None None] MFA Rules not processed for user `140059c5b30c426ab6d7f1236eb5f684`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:39:59.397637 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-9275362a-9952-4f99-b4b3-f23480abfd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:39:59.451041 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1138/2258] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:39:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:40:05.720175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eddba8-8b3a-4534-af8f-deeba3995ed3 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:40:05.721172 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eddba8-8b3a-4534-af8f-deeba3995ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:40:05.721172 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eddba8-8b3a-4534-af8f-deeba3995ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:40:05.739903 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-36eddba8-8b3a-4534-af8f-deeba3995ed3 None None] MFA Rules not processed for user `00f17de515184c01a73c454c7f0b7e70`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:40:05.741164 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-36eddba8-8b3a-4534-af8f-deeba3995ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:40:05.796154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1121/2259] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:40:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:40:09.444534 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=25 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:40:15.789401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=25 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:40:39.135055 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20684261-1871-4172-9137-5ebd4e91eac1 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:40:39.135627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20684261-1871-4172-9137-5ebd4e91eac1 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:40:39.135627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20684261-1871-4172-9137-5ebd4e91eac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:40:39.154858 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-20684261-1871-4172-9137-5ebd4e91eac1 None None] MFA Rules not processed for user `1c0838069cba4d1389ec6fc809db70a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:40:39.156073 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-20684261-1871-4172-9137-5ebd4e91eac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:40:39.211290 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1139/2260] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:40:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:40:47.788400 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eddba8-8b3a-4534-af8f-deeba3995ed3 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:40:47.803654 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eddba8-8b3a-4534-af8f-deeba3995ed3 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:40:47.812414 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f9eb1-3839-4eed-963a-d2541407809b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:40:47.815512 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f9eb1-3839-4eed-963a-d2541407809b service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:40:47.821980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f9eb1-3839-4eed-963a-d2541407809b service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:40:47.824883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f9eb1-3839-4eed-963a-d2541407809b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:40:47.827181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ad1f9eb1-3839-4eed-963a-d2541407809b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:40:47.861257 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f9eb1-3839-4eed-963a-d2541407809b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:40:47.866756 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f9eb1-3839-4eed-963a-d2541407809b service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:40:47.880492 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1122/2261] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:40:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:40:49.204501 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=25 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:40:49.428040 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9da3b-18b7-45fb-a324-b49d6914ab37 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:40:49.428400 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9da3b-18b7-45fb-a324-b49d6914ab37 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:40:49.428752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9da3b-18b7-45fb-a324-b49d6914ab37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:40:49.449026 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-96b9da3b-18b7-45fb-a324-b49d6914ab37 None None] MFA Rules not processed for user `22bb7775bae64374b11462a53853e54f`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:40:49.450647 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-96b9da3b-18b7-45fb-a324-b49d6914ab37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:40:49.508577 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1140/2262] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:40:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:40:52.353570 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f9eb1-3839-4eed-963a-d2541407809b service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:40:52.372978 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f9eb1-3839-4eed-963a-d2541407809b service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:40:52.377875 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb8ee9-633e-40e7-80ff-87c4fba9f4d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:40:52.378502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb8ee9-633e-40e7-80ff-87c4fba9f4d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:40:52.378812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb8ee9-633e-40e7-80ff-87c4fba9f4d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:40:52.379070 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb8ee9-633e-40e7-80ff-87c4fba9f4d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:40:52.380987 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-41fb8ee9-633e-40e7-80ff-87c4fba9f4d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:40:52.396125 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fb8ee9-633e-40e7-80ff-87c4fba9f4d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:40:52.397516 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fb8ee9-633e-40e7-80ff-87c4fba9f4d8 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:40:52.410665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1123/2263] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:40:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:40:59.501775 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=25 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:41:02.403934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=12 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:41:03.064340 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9da3b-18b7-45fb-a324-b49d6914ab37 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:03.065789 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-96b9da3b-18b7-45fb-a324-b49d6914ab37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:03.082696 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9da3b-18b7-45fb-a324-b49d6914ab37 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:03.088136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe07fc-439a-4bd4-ae02-e2bdb4a490f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:03.088746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe07fc-439a-4bd4-ae02-e2bdb4a490f0 service nova] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:03.089070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe07fc-439a-4bd4-ae02-e2bdb4a490f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:03.089348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe07fc-439a-4bd4-ae02-e2bdb4a490f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:03.091038 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-5afe07fc-439a-4bd4-ae02-e2bdb4a490f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:03.101445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe07fc-439a-4bd4-ae02-e2bdb4a490f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:03.103035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe07fc-439a-4bd4-ae02-e2bdb4a490f0 service nova] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:03.112034 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1141/2264] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:41:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:41:03.550408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb8ee9-633e-40e7-80ff-87c4fba9f4d8 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:03.551858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-41fb8ee9-633e-40e7-80ff-87c4fba9f4d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:03.607566 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb8ee9-633e-40e7-80ff-87c4fba9f4d8 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:03.611772 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875000f9-3ce4-4122-bbd7-f16c37e59308 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:03.612381 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875000f9-3ce4-4122-bbd7-f16c37e59308 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:03.612663 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875000f9-3ce4-4122-bbd7-f16c37e59308 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:03.612973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875000f9-3ce4-4122-bbd7-f16c37e59308 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:03.614704 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-875000f9-3ce4-4122-bbd7-f16c37e59308 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:03.625327 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875000f9-3ce4-4122-bbd7-f16c37e59308 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:03.626788 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875000f9-3ce4-4122-bbd7-f16c37e59308 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:03.635593 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1124/2265] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:41:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:41:10.575118 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe07fc-439a-4bd4-ae02-e2bdb4a490f0 service nova] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:10.576512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-5afe07fc-439a-4bd4-ae02-e2bdb4a490f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:10.591283 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe07fc-439a-4bd4-ae02-e2bdb4a490f0 service nova] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:10.596661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e765cb-8790-40cc-9e3f-996517d797b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f8e0f82a8541aead6cd8cc45f4a86b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:10.597324 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e765cb-8790-40cc-9e3f-996517d797b2 service placement] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:10.597598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e765cb-8790-40cc-9e3f-996517d797b2 service placement] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:10.597866 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e765cb-8790-40cc-9e3f-996517d797b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:10.599762 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-36e765cb-8790-40cc-9e3f-996517d797b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:10.612469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e765cb-8790-40cc-9e3f-996517d797b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:10.614025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e765cb-8790-40cc-9e3f-996517d797b2 service placement] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:10.626208 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1142/2266] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:41:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:41:10.643598 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875000f9-3ce4-4122-bbd7-f16c37e59308 service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:10.645133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-875000f9-3ce4-4122-bbd7-f16c37e59308 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:10.714259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875000f9-3ce4-4122-bbd7-f16c37e59308 service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:10.719129 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459820bc-d69c-4458-a040-9f0a48299abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:10.719756 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459820bc-d69c-4458-a040-9f0a48299abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:10.720034 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459820bc-d69c-4458-a040-9f0a48299abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:10.720307 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459820bc-d69c-4458-a040-9f0a48299abf admin admin] PATH_INFO: `/v3/users/00f17de515184c01a73c454c7f0b7e70` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:10.721472 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459820bc-d69c-4458-a040-9f0a48299abf admin admin] RBAC: Authorizing `identity:delete_user(user_id=00f17de515184c01a73c454c7f0b7e70)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:10.723383 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459820bc-d69c-4458-a040-9f0a48299abf admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:10.769290 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-459820bc-d69c-4458-a040-9f0a48299abf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00f17de515184c01a73c454c7f0b7e70'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:41:10.776252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-459820bc-d69c-4458-a040-9f0a48299abf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00f17de515184c01a73c454c7f0b7e70'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:41:10.782980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-459820bc-d69c-4458-a040-9f0a48299abf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00f17de515184c01a73c454c7f0b7e70'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:41:10.794576 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-459820bc-d69c-4458-a040-9f0a48299abf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00f17de515184c01a73c454c7f0b7e70'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:41:10.795204 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-459820bc-d69c-4458-a040-9f0a48299abf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00f17de515184c01a73c454c7f0b7e70'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:41:10.801728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1125/2267] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:41:10 2022] DELETE /identity/v3/users/00f17de515184c01a73c454c7f0b7e70 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:41:10.974801 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e765cb-8790-40cc-9e3f-996517d797b2 service placement] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:10.986232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e765cb-8790-40cc-9e3f-996517d797b2 service placement] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:11.052039 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff454c0-f122-4c57-9e21-653c70fc2565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:11.052694 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff454c0-f122-4c57-9e21-653c70fc2565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:11.053002 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff454c0-f122-4c57-9e21-653c70fc2565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:11.053273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff454c0-f122-4c57-9e21-653c70fc2565 admin admin] PATH_INFO: `/v3/projects/ce1f193c68ef49b78c95c02f7413b7d6` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:11.054415 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff454c0-f122-4c57-9e21-653c70fc2565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce1f193c68ef49b78c95c02f7413b7d6)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:11.056426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff454c0-f122-4c57-9e21-653c70fc2565 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:11.104767 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1ff454c0-f122-4c57-9e21-653c70fc2565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce1f193c68ef49b78c95c02f7413b7d6', 'request_id': 'req-1ff454c0-f122-4c57-9e21-653c70fc2565'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:41:11.113372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1ff454c0-f122-4c57-9e21-653c70fc2565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce1f193c68ef49b78c95c02f7413b7d6', 'request_id': 'req-1ff454c0-f122-4c57-9e21-653c70fc2565'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:41:11.119590 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1ff454c0-f122-4c57-9e21-653c70fc2565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce1f193c68ef49b78c95c02f7413b7d6', 'request_id': 'req-1ff454c0-f122-4c57-9e21-653c70fc2565'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:41:11.132084 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1ff454c0-f122-4c57-9e21-653c70fc2565 admin admin] The token cache is being invalidate because project ce1f193c68ef49b78c95c02f7413b7d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:41:11.132369 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1ff454c0-f122-4c57-9e21-653c70fc2565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:41:11.133036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1ff454c0-f122-4c57-9e21-653c70fc2565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:41:11.134527 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1143/2268] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:41:10 2022] DELETE /identity/v3/projects/ce1f193c68ef49b78c95c02f7413b7d6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:41:11.139387 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14503115-9f13-4e4c-91a3-56f3cde36424 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:11.139674 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14503115-9f13-4e4c-91a3-56f3cde36424 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:11.139956 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14503115-9f13-4e4c-91a3-56f3cde36424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:11.161476 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-14503115-9f13-4e4c-91a3-56f3cde36424 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:41:11.162616 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-14503115-9f13-4e4c-91a3-56f3cde36424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:11.180487 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:41:11.225082 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1126/2269] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:41:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:41:11.233668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d068988-8d28-4be6-a329-9bfb025c1c24 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:11.233988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d068988-8d28-4be6-a329-9bfb025c1c24 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:11.234843 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d068988-8d28-4be6-a329-9bfb025c1c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:11.254004 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-9d068988-8d28-4be6-a329-9bfb025c1c24 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:41:11.255197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-9d068988-8d28-4be6-a329-9bfb025c1c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:11.263015 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1144/2270] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:41:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:41:11.267136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14503115-9f13-4e4c-91a3-56f3cde36424 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:11.276805 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14503115-9f13-4e4c-91a3-56f3cde36424 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:11.281867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e378ec2-8e53-438a-8366-6c2c5277cd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:11.282468 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e378ec2-8e53-438a-8366-6c2c5277cd00 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:11.282750 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e378ec2-8e53-438a-8366-6c2c5277cd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:11.283023 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e378ec2-8e53-438a-8366-6c2c5277cd00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:11.283710 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e378ec2-8e53-438a-8366-6c2c5277cd00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:41:11.284015 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e378ec2-8e53-438a-8366-6c2c5277cd00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:11.285642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e378ec2-8e53-438a-8366-6c2c5277cd00 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:11.297497 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1127/2271] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:41:11.302023 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d068988-8d28-4be6-a329-9bfb025c1c24 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:11.311316 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d068988-8d28-4be6-a329-9bfb025c1c24 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:11.315972 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68920c0b-d5ed-4c8c-ac8d-75642628a397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:11.316579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68920c0b-d5ed-4c8c-ac8d-75642628a397 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:11.316861 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68920c0b-d5ed-4c8c-ac8d-75642628a397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:11.317175 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68920c0b-d5ed-4c8c-ac8d-75642628a397 admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:11.317840 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68920c0b-d5ed-4c8c-ac8d-75642628a397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:11.319747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68920c0b-d5ed-4c8c-ac8d-75642628a397 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:11.348204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1145/2272] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:41:11 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:41:11.352266 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e378ec2-8e53-438a-8366-6c2c5277cd00 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:11.362887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e378ec2-8e53-438a-8366-6c2c5277cd00 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:11.419178 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c67c951-5219-4078-977a-083c6df0c481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:11.419949 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67c951-5219-4078-977a-083c6df0c481 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:11.420230 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67c951-5219-4078-977a-083c6df0c481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:11.420503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67c951-5219-4078-977a-083c6df0c481 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:11.421240 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c67c951-5219-4078-977a-083c6df0c481 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:11.433085 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c67c951-5219-4078-977a-083c6df0c481 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:11.449852 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1128/2273] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:41:11 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:41:11.450860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68920c0b-d5ed-4c8c-ac8d-75642628a397 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:11.464198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68920c0b-d5ed-4c8c-ac8d-75642628a397 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:11.469580 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e394f32-f32f-4cb3-8af6-083cd61bc4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:11.470243 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e394f32-f32f-4cb3-8af6-083cd61bc4cd admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:11.470516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e394f32-f32f-4cb3-8af6-083cd61bc4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:11.470790 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e394f32-f32f-4cb3-8af6-083cd61bc4cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:11.471483 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e394f32-f32f-4cb3-8af6-083cd61bc4cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:11.473219 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e394f32-f32f-4cb3-8af6-083cd61bc4cd admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:11.484464 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1146/2274] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:41:11.488637 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c67c951-5219-4078-977a-083c6df0c481 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:11.499460 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c67c951-5219-4078-977a-083c6df0c481 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:11.508650 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb797f0-d5bc-4bd2-b6e0-dfe146a87846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:11.509138 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb797f0-d5bc-4bd2-b6e0-dfe146a87846 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:11.509421 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb797f0-d5bc-4bd2-b6e0-dfe146a87846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:11.509697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb797f0-d5bc-4bd2-b6e0-dfe146a87846 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:11.510399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb797f0-d5bc-4bd2-b6e0-dfe146a87846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:11.512003 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb797f0-d5bc-4bd2-b6e0-dfe146a87846 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:11.524013 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1129/2275] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:41:11.528315 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e394f32-f32f-4cb3-8af6-083cd61bc4cd admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:11.539062 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e394f32-f32f-4cb3-8af6-083cd61bc4cd admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:11.543791 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14685d65-e0f5-4d86-8887-9e45bb0147d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:11.544459 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14685d65-e0f5-4d86-8887-9e45bb0147d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:11.544751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14685d65-e0f5-4d86-8887-9e45bb0147d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:11.545071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14685d65-e0f5-4d86-8887-9e45bb0147d6 admin admin] PATH_INFO: `/v3/projects/95d1afc492b44c07800f8d7288dac741/users/6e5020300565412f9a251e471aeb17b9/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:11.560288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14685d65-e0f5-4d86-8887-9e45bb0147d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95d1afc492b44c07800f8d7288dac741, user_id=6e5020300565412f9a251e471aeb17b9, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:11.563107 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14685d65-e0f5-4d86-8887-9e45bb0147d6 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:11.572393 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:41:11.578543 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1147/2276] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:41:11 2022] PUT /identity/v3/projects/95d1afc492b44c07800f8d7288dac741/users/6e5020300565412f9a251e471aeb17b9/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:41:11.583674 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937adefb-98f2-493f-9159-6771dc71f02f None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:11.583983 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937adefb-98f2-493f-9159-6771dc71f02f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:11.584452 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937adefb-98f2-493f-9159-6771dc71f02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:11.604907 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-937adefb-98f2-493f-9159-6771dc71f02f None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:41:11.606027 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-937adefb-98f2-493f-9159-6771dc71f02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:11.669189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1130/2277] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:41:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:41:11.679031 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14685d65-e0f5-4d86-8887-9e45bb0147d6 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:11.681000 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-14685d65-e0f5-4d86-8887-9e45bb0147d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:11.738846 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14685d65-e0f5-4d86-8887-9e45bb0147d6 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:11.742517 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc6d34-080b-47bc-9c25-df4c54fea83b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:11.743137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc6d34-080b-47bc-9c25-df4c54fea83b service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:11.743414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc6d34-080b-47bc-9c25-df4c54fea83b service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:11.743691 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc6d34-080b-47bc-9c25-df4c54fea83b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:11.752757 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc6d34-080b-47bc-9c25-df4c54fea83b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:11.754234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc6d34-080b-47bc-9c25-df4c54fea83b service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:11.764446 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1148/2278] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:41:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:41:16.423832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b66689-3f8f-4e79-94f6-9f61c0bdb3a5 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:16.424326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b66689-3f8f-4e79-94f6-9f61c0bdb3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:16.424532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b66689-3f8f-4e79-94f6-9f61c0bdb3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:16.453876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-53b66689-3f8f-4e79-94f6-9f61c0bdb3a5 None None] MFA Rules not processed for user `6e5020300565412f9a251e471aeb17b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:41:16.455103 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-53b66689-3f8f-4e79-94f6-9f61c0bdb3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:16.472437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:41:16.525337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1131/2279] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:41:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:41:16.530005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc6d34-080b-47bc-9c25-df4c54fea83b service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:16.545565 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc6d34-080b-47bc-9c25-df4c54fea83b service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:16.550979 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8114774-81c8-43fc-a481-931a2191e3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:16.551767 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8114774-81c8-43fc-a481-931a2191e3ce admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:16.552144 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8114774-81c8-43fc-a481-931a2191e3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:16.552521 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8114774-81c8-43fc-a481-931a2191e3ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:16.553464 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8114774-81c8-43fc-a481-931a2191e3ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:16.555591 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8114774-81c8-43fc-a481-931a2191e3ce admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:16.604111 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b66689-3f8f-4e79-94f6-9f61c0bdb3a5 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:16.605902 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1149/2280] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:41:16 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:41:16.610551 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8114774-81c8-43fc-a481-931a2191e3ce admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:16.615965 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-53b66689-3f8f-4e79-94f6-9f61c0bdb3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:16.751170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8114774-81c8-43fc-a481-931a2191e3ce admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:16.768677 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a237484-2208-4ff5-91ca-d0db08616287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:16.769504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a237484-2208-4ff5-91ca-d0db08616287 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:16.769845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a237484-2208-4ff5-91ca-d0db08616287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:16.770129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a237484-2208-4ff5-91ca-d0db08616287 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:16.770807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a237484-2208-4ff5-91ca-d0db08616287 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:16.772919 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a237484-2208-4ff5-91ca-d0db08616287 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:16.811526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1150/2281] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:41:16 2022] POST /identity/v3/users => generated 299 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:41:16.818409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a237484-2208-4ff5-91ca-d0db08616287 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:16.820842 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b66689-3f8f-4e79-94f6-9f61c0bdb3a5 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:16.830786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a237484-2208-4ff5-91ca-d0db08616287 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:16.837274 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe159589-e7c4-491e-bf2b-396fee4ba7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:16.837900 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe159589-e7c4-491e-bf2b-396fee4ba7a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:16.838183 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe159589-e7c4-491e-bf2b-396fee4ba7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:16.838459 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe159589-e7c4-491e-bf2b-396fee4ba7a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:16.839157 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe159589-e7c4-491e-bf2b-396fee4ba7a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:16.840757 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe159589-e7c4-491e-bf2b-396fee4ba7a6 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:16.860923 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1151/2282] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:41:16 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:41:16.864780 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe159589-e7c4-491e-bf2b-396fee4ba7a6 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:16.881184 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe159589-e7c4-491e-bf2b-396fee4ba7a6 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:16.887563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a2ef1-e0d7-403c-9bf0-2ff8eefeb237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:16.888367 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a2ef1-e0d7-403c-9bf0-2ff8eefeb237 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:16.888720 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a2ef1-e0d7-403c-9bf0-2ff8eefeb237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:16.889104 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a2ef1-e0d7-403c-9bf0-2ff8eefeb237 admin admin] PATH_INFO: `/v3/projects/49fb5ac1be6848ec9263b3431bd87f4a/users/684e37a8185a4199b95d407f9d979a06/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:16.905705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a2ef1-e0d7-403c-9bf0-2ff8eefeb237 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49fb5ac1be6848ec9263b3431bd87f4a, user_id=684e37a8185a4199b95d407f9d979a06, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:16.908792 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230c7b9-f32b-4802-9883-08543d897fdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:16.909116 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a2ef1-e0d7-403c-9bf0-2ff8eefeb237 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:16.909807 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230c7b9-f32b-4802-9883-08543d897fdb service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:16.910240 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230c7b9-f32b-4802-9883-08543d897fdb service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:16.910662 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230c7b9-f32b-4802-9883-08543d897fdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:16.913293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d230c7b9-f32b-4802-9883-08543d897fdb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:16.923468 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1152/2283] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:41:16 2022] PUT /identity/v3/projects/49fb5ac1be6848ec9263b3431bd87f4a/users/684e37a8185a4199b95d407f9d979a06/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:41:16.927662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a2ef1-e0d7-403c-9bf0-2ff8eefeb237 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:16.929007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230c7b9-f32b-4802-9883-08543d897fdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:16.930878 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230c7b9-f32b-4802-9883-08543d897fdb service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:16.942806 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1132/2284] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:41:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:41:16.978886 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:41:17.017388 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a2ef1-e0d7-403c-9bf0-2ff8eefeb237 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:17.023137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c221df-11a8-4c6b-8edc-cae3d8cc9b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:17.023937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c221df-11a8-4c6b-8edc-cae3d8cc9b8a admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:17.024222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c221df-11a8-4c6b-8edc-cae3d8cc9b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:17.024500 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c221df-11a8-4c6b-8edc-cae3d8cc9b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:17.026574 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c221df-11a8-4c6b-8edc-cae3d8cc9b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:17.027373 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c221df-11a8-4c6b-8edc-cae3d8cc9b8a admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:17.040353 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1153/2285] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:41:16 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:41:17.044209 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230c7b9-f32b-4802-9883-08543d897fdb service nova] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:17.059625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230c7b9-f32b-4802-9883-08543d897fdb service nova] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:17.064414 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f77ae-950d-436b-a23a-c20d45625212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:17.065298 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f77ae-950d-436b-a23a-c20d45625212 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:17.065697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f77ae-950d-436b-a23a-c20d45625212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:17.066068 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f77ae-950d-436b-a23a-c20d45625212 admin admin] PATH_INFO: `/v3/projects/49fb5ac1be6848ec9263b3431bd87f4a/users/684e37a8185a4199b95d407f9d979a06/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:17.068191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f77ae-950d-436b-a23a-c20d45625212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49fb5ac1be6848ec9263b3431bd87f4a, user_id=684e37a8185a4199b95d407f9d979a06, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:17.071066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f77ae-950d-436b-a23a-c20d45625212 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:17.096724 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1133/2286] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:41:17 2022] PUT /identity/v3/projects/49fb5ac1be6848ec9263b3431bd87f4a/users/684e37a8185a4199b95d407f9d979a06/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:41:21.084598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356f1ef-73d6-4da2-a20e-f2e141deae62 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:21.085160 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356f1ef-73d6-4da2-a20e-f2e141deae62 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:21.085160 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356f1ef-73d6-4da2-a20e-f2e141deae62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:21.106610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-8356f1ef-73d6-4da2-a20e-f2e141deae62 None None] MFA Rules not processed for user `140059c5b30c426ab6d7f1236eb5f684`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:41:21.108273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-8356f1ef-73d6-4da2-a20e-f2e141deae62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:21.217357 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1154/2287] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:41:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:41:21.652591 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f77ae-950d-436b-a23a-c20d45625212 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:21.654411 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-718f77ae-950d-436b-a23a-c20d45625212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:21.712114 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:41:21.758176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f77ae-950d-436b-a23a-c20d45625212 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:21.764226 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1ba060-f4a5-4ae4-a877-5f2520de5aef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:21.766252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1ba060-f4a5-4ae4-a877-5f2520de5aef service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:21.769620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1ba060-f4a5-4ae4-a877-5f2520de5aef service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:21.770380 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1ba060-f4a5-4ae4-a877-5f2520de5aef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:21.773164 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-7a1ba060-f4a5-4ae4-a877-5f2520de5aef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:21.787627 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1ba060-f4a5-4ae4-a877-5f2520de5aef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:21.789752 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1ba060-f4a5-4ae4-a877-5f2520de5aef service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:21.822201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1134/2288] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:41:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:41:22.251523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fc0a3-00f0-4d82-9dea-226ccdfddeb8 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:22.251947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fc0a3-00f0-4d82-9dea-226ccdfddeb8 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:22.252104 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fc0a3-00f0-4d82-9dea-226ccdfddeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:22.310074 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1ba060-f4a5-4ae4-a877-5f2520de5aef service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:22.310358 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-539fc0a3-00f0-4d82-9dea-226ccdfddeb8 None None] MFA Rules not processed for user `684e37a8185a4199b95d407f9d979a06`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:41:22.311799 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-539fc0a3-00f0-4d82-9dea-226ccdfddeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:22.313486 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-7a1ba060-f4a5-4ae4-a877-5f2520de5aef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:22.403365 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1ba060-f4a5-4ae4-a877-5f2520de5aef service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:22.409043 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3281bce-5858-41bf-8297-333885785259 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f8e0f82a8541aead6cd8cc45f4a86b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:22.409691 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3281bce-5858-41bf-8297-333885785259 service placement] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:22.409975 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3281bce-5858-41bf-8297-333885785259 service placement] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:22.410256 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3281bce-5858-41bf-8297-333885785259 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:22.414294 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c3281bce-5858-41bf-8297-333885785259 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:22.422132 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1155/2289] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:41:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:41:22.429226 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3281bce-5858-41bf-8297-333885785259 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:22.430838 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3281bce-5858-41bf-8297-333885785259 service placement] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:22.434745 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fc0a3-00f0-4d82-9dea-226ccdfddeb8 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:22.436366 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-539fc0a3-00f0-4d82-9dea-226ccdfddeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:22.442774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1135/2290] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:41:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:41:22.499488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fc0a3-00f0-4d82-9dea-226ccdfddeb8 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:22.504291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ca522-f23e-4aeb-aae6-f09b135a65b1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c50c2d1781b04a5fa8be81bce9205747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:22.505105 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ca522-f23e-4aeb-aae6-f09b135a65b1 service glance] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:22.505396 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ca522-f23e-4aeb-aae6-f09b135a65b1 service glance] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:22.505676 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ca522-f23e-4aeb-aae6-f09b135a65b1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:22.519325 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:41:22.579956 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ca522-f23e-4aeb-aae6-f09b135a65b1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:22.581976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ca522-f23e-4aeb-aae6-f09b135a65b1 service glance] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:22.595817 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1156/2291] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:41:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:41:22.747917 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1b6b9-f665-4c5b-a7ce-4f876d5da09c None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:22.748768 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1b6b9-f665-4c5b-a7ce-4f876d5da09c None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:22.749231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1b6b9-f665-4c5b-a7ce-4f876d5da09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:22.782163 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-4fd1b6b9-f665-4c5b-a7ce-4f876d5da09c None None] MFA Rules not processed for user `07d6bffc69674103b4c6540598a780b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:41:22.784511 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-4fd1b6b9-f665-4c5b-a7ce-4f876d5da09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:22.861007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1136/2292] 10.222.0.74 () {58 vars in 1009 bytes} [Wed Jul 27 16:41:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:41:22.915766 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ca522-f23e-4aeb-aae6-f09b135a65b1 service glance] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:22.917986 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-5a9ca522-f23e-4aeb-aae6-f09b135a65b1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:23.040620 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ca522-f23e-4aeb-aae6-f09b135a65b1 service glance] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:23.047938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736cc25-437e-4fa1-95a0-6ff1c3b9957f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb45ded9f75e4fa2973b564f83bdfaf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:23.048446 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736cc25-437e-4fa1-95a0-6ff1c3b9957f service swift] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:23.048791 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736cc25-437e-4fa1-95a0-6ff1c3b9957f service swift] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:23.049102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736cc25-437e-4fa1-95a0-6ff1c3b9957f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:23.060919 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0736cc25-437e-4fa1-95a0-6ff1c3b9957f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:23.062628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0736cc25-437e-4fa1-95a0-6ff1c3b9957f service swift] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:23.071741 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1157/2293] 10.222.0.74 () {60 vars in 1384 bytes} [Wed Jul 27 16:41:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 16:41:24.738461 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1b6b9-f665-4c5b-a7ce-4f876d5da09c None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:24.749409 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1b6b9-f665-4c5b-a7ce-4f876d5da09c None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:24.762088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f68fc-3ce3-4d67-ae82-b9e25de67951 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:24.763182 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f68fc-3ce3-4d67-ae82-b9e25de67951 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:24.763686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f68fc-3ce3-4d67-ae82-b9e25de67951 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:24.765592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f68fc-3ce3-4d67-ae82-b9e25de67951 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:24.769896 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-a40f68fc-3ce3-4d67-ae82-b9e25de67951 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:24.781979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f68fc-3ce3-4d67-ae82-b9e25de67951 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:24.783459 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f68fc-3ce3-4d67-ae82-b9e25de67951 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:24.794945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1137/2294] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:41:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:41:26.178316 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736cc25-437e-4fa1-95a0-6ff1c3b9957f service swift] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:26.179866 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-0736cc25-437e-4fa1-95a0-6ff1c3b9957f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:26.255316 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736cc25-437e-4fa1-95a0-6ff1c3b9957f service swift] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:26.260248 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa74de-1b24-472a-98f8-1d49f65d5a46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:26.261023 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa74de-1b24-472a-98f8-1d49f65d5a46 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:26.261406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa74de-1b24-472a-98f8-1d49f65d5a46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:26.261778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa74de-1b24-472a-98f8-1d49f65d5a46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:26.264954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-95aa74de-1b24-472a-98f8-1d49f65d5a46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:26.339068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa74de-1b24-472a-98f8-1d49f65d5a46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:26.341119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa74de-1b24-472a-98f8-1d49f65d5a46 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:26.349629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:41:26.357219 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1158/2295] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:41:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:41:28.998387 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f68fc-3ce3-4d67-ae82-b9e25de67951 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:29.000366 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-a40f68fc-3ce3-4d67-ae82-b9e25de67951 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:29.022196 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f68fc-3ce3-4d67-ae82-b9e25de67951 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:29.047567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a39bd0-cc5b-4ef7-bf19-7e845adca38c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:29.049062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a39bd0-cc5b-4ef7-bf19-7e845adca38c service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:29.049355 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a39bd0-cc5b-4ef7-bf19-7e845adca38c service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:29.049637 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a39bd0-cc5b-4ef7-bf19-7e845adca38c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:29.051465 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-e9a39bd0-cc5b-4ef7-bf19-7e845adca38c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:29.121904 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:41:29.136017 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a39bd0-cc5b-4ef7-bf19-7e845adca38c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:29.137619 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a39bd0-cc5b-4ef7-bf19-7e845adca38c service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:29.158349 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1138/2296] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:41:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:41:33.386199 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72728da-9cc0-42ab-bb4a-f9187872b19a None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:33.386676 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72728da-9cc0-42ab-bb4a-f9187872b19a None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:33.386765 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72728da-9cc0-42ab-bb4a-f9187872b19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:33.406460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-e72728da-9cc0-42ab-bb4a-f9187872b19a None None] MFA Rules not processed for user `6e5020300565412f9a251e471aeb17b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:41:33.407742 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e72728da-9cc0-42ab-bb4a-f9187872b19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:33.415107 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1159/2297] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:41:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:41:36.287767 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9c222-d70d-4c8c-a398-ce3b30991934 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:36.288767 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9c222-d70d-4c8c-a398-ce3b30991934 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:36.289370 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9c222-d70d-4c8c-a398-ce3b30991934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:36.313706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-c9b9c222-d70d-4c8c-a398-ce3b30991934 None None] MFA Rules not processed for user `07d6bffc69674103b4c6540598a780b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:41:36.315347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c9b9c222-d70d-4c8c-a398-ce3b30991934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:36.324102 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1139/2298] 10.222.0.74 () {58 vars in 1009 bytes} [Wed Jul 27 16:41:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:41:36.340253 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72728da-9cc0-42ab-bb4a-f9187872b19a None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:36.364784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72728da-9cc0-42ab-bb4a-f9187872b19a None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:36.383416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a20c7-3a36-4c82-aae7-4dda0fd79ace service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb45ded9f75e4fa2973b564f83bdfaf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:36.384090 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a20c7-3a36-4c82-aae7-4dda0fd79ace service swift] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:36.384380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a20c7-3a36-4c82-aae7-4dda0fd79ace service swift] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:36.384656 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a20c7-3a36-4c82-aae7-4dda0fd79ace service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:36.400144 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a20c7-3a36-4c82-aae7-4dda0fd79ace service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:36.401820 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a20c7-3a36-4c82-aae7-4dda0fd79ace service swift] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:36.425834 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1160/2299] 10.222.0.74 () {60 vars in 1384 bytes} [Wed Jul 27 16:41:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 16:41:36.787935 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9c222-d70d-4c8c-a398-ce3b30991934 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:36.896134 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9c222-d70d-4c8c-a398-ce3b30991934 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:36.908230 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64654469-8075-45f9-aef0-97c1ee7ea3a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:36.909297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64654469-8075-45f9-aef0-97c1ee7ea3a5 service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:36.909946 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64654469-8075-45f9-aef0-97c1ee7ea3a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:36.911402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64654469-8075-45f9-aef0-97c1ee7ea3a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:36.915659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-64654469-8075-45f9-aef0-97c1ee7ea3a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:36.943174 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64654469-8075-45f9-aef0-97c1ee7ea3a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:36.946275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64654469-8075-45f9-aef0-97c1ee7ea3a5 service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:36.982370 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1140/2300] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:41:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:41:46.415879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=9 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:41:46.956343 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=41 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:41:49.149872 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bed76-ab70-4f50-b24c-ef9a34283172 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:49.150312 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bed76-ab70-4f50-b24c-ef9a34283172 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:49.150419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bed76-ab70-4f50-b24c-ef9a34283172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:49.170635 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-804bed76-ab70-4f50-b24c-ef9a34283172 None None] MFA Rules not processed for user `7a0bd78165c94d9e9750fe4af4f7d375`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:41:49.172297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-804bed76-ab70-4f50-b24c-ef9a34283172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:49.181381 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1161/2301] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:41:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:41:50.574103 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64654469-8075-45f9-aef0-97c1ee7ea3a5 service nova] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:41:50.575537 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-64654469-8075-45f9-aef0-97c1ee7ea3a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:50.590364 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64654469-8075-45f9-aef0-97c1ee7ea3a5 service nova] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:41:50.596031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c722d7-b387-4239-aa1d-65e974e8036d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:41:50.596650 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c722d7-b387-4239-aa1d-65e974e8036d service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:41:50.596954 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c722d7-b387-4239-aa1d-65e974e8036d service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:41:50.597230 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c722d7-b387-4239-aa1d-65e974e8036d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:41:50.598977 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-90c722d7-b387-4239-aa1d-65e974e8036d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:41:50.611524 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c722d7-b387-4239-aa1d-65e974e8036d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:41:50.613154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c722d7-b387-4239-aa1d-65e974e8036d service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:41:50.624395 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1141/2302] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:41:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:41:59.168083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:00.175500 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bed76-ab70-4f50-b24c-ef9a34283172 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:00.185688 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bed76-ab70-4f50-b24c-ef9a34283172 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:00.189392 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0752b61-df00-47c4-969b-e379c78848ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:00.190007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0752b61-df00-47c4-969b-e379c78848ca service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:00.190285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0752b61-df00-47c4-969b-e379c78848ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:00.190555 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0752b61-df00-47c4-969b-e379c78848ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:00.192257 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-c0752b61-df00-47c4-969b-e379c78848ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:00.255857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0752b61-df00-47c4-969b-e379c78848ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:00.257624 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0752b61-df00-47c4-969b-e379c78848ca service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:00.268204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1162/2303] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:42:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:42:00.620076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=6 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:03.713902 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c722d7-b387-4239-aa1d-65e974e8036d service nova] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:03.725418 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c722d7-b387-4239-aa1d-65e974e8036d service nova] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:03.730339 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddde342-ac09-4363-b524-95ac2e7e3127 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f8e0f82a8541aead6cd8cc45f4a86b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:03.730939 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddde342-ac09-4363-b524-95ac2e7e3127 service placement] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:03.731404 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddde342-ac09-4363-b524-95ac2e7e3127 service placement] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:03.731674 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddde342-ac09-4363-b524-95ac2e7e3127 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:03.734016 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0ddde342-ac09-4363-b524-95ac2e7e3127 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:03.745538 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddde342-ac09-4363-b524-95ac2e7e3127 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:03.747139 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddde342-ac09-4363-b524-95ac2e7e3127 service placement] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:03.758204 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1142/2304] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:42:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:42:03.768364 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0752b61-df00-47c4-969b-e379c78848ca service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:03.769917 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-c0752b61-df00-47c4-969b-e379c78848ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:03.784591 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0752b61-df00-47c4-969b-e379c78848ca service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:03.789649 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7420d-76c8-472a-a3d3-c18517a927ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:03.790298 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7420d-76c8-472a-a3d3-c18517a927ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:03.790598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7420d-76c8-472a-a3d3-c18517a927ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:03.790866 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7420d-76c8-472a-a3d3-c18517a927ff admin admin] PATH_INFO: `/v3/users/1c0838069cba4d1389ec6fc809db70a7` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:03.792039 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7420d-76c8-472a-a3d3-c18517a927ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c0838069cba4d1389ec6fc809db70a7)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:03.793997 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7420d-76c8-472a-a3d3-c18517a927ff admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:03.842971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-b0a7420d-76c8-472a-a3d3-c18517a927ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c0838069cba4d1389ec6fc809db70a7'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:03.849747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-b0a7420d-76c8-472a-a3d3-c18517a927ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c0838069cba4d1389ec6fc809db70a7'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:03.862369 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-b0a7420d-76c8-472a-a3d3-c18517a927ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c0838069cba4d1389ec6fc809db70a7'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:03.869527 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-b0a7420d-76c8-472a-a3d3-c18517a927ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c0838069cba4d1389ec6fc809db70a7'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:03.870708 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-b0a7420d-76c8-472a-a3d3-c18517a927ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c0838069cba4d1389ec6fc809db70a7'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:03.877351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1163/2305] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:42:03 2022] DELETE /identity/v3/users/1c0838069cba4d1389ec6fc809db70a7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:04.050408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddde342-ac09-4363-b524-95ac2e7e3127 service placement] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:04.062371 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddde342-ac09-4363-b524-95ac2e7e3127 service placement] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:04.123193 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01997a31-9339-4632-af90-1e0857ef0550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:04.123868 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01997a31-9339-4632-af90-1e0857ef0550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:04.124150 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01997a31-9339-4632-af90-1e0857ef0550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:04.124425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01997a31-9339-4632-af90-1e0857ef0550 admin admin] PATH_INFO: `/v3/projects/027a003da80c43ca9b4b337527b93d3a` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:04.125611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01997a31-9339-4632-af90-1e0857ef0550 admin admin] RBAC: Authorizing `identity:delete_project(project_id=027a003da80c43ca9b4b337527b93d3a)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:04.127554 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01997a31-9339-4632-af90-1e0857ef0550 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:04.194269 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-01997a31-9339-4632-af90-1e0857ef0550 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '027a003da80c43ca9b4b337527b93d3a', 'request_id': 'req-01997a31-9339-4632-af90-1e0857ef0550'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:04.200928 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-01997a31-9339-4632-af90-1e0857ef0550 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '027a003da80c43ca9b4b337527b93d3a', 'request_id': 'req-01997a31-9339-4632-af90-1e0857ef0550'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:04.209276 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-01997a31-9339-4632-af90-1e0857ef0550 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '027a003da80c43ca9b4b337527b93d3a', 'request_id': 'req-01997a31-9339-4632-af90-1e0857ef0550'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:04.222678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-01997a31-9339-4632-af90-1e0857ef0550 admin admin] The token cache is being invalidate because project 027a003da80c43ca9b4b337527b93d3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:42:04.223245 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-01997a31-9339-4632-af90-1e0857ef0550 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:04.223996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-01997a31-9339-4632-af90-1e0857ef0550 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:04.226046 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1143/2306] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:42:04 2022] DELETE /identity/v3/projects/027a003da80c43ca9b4b337527b93d3a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:04.230755 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5b74c-5df7-42c5-94d2-517c74d2284c None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:04.231186 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5b74c-5df7-42c5-94d2-517c74d2284c None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:04.231302 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5b74c-5df7-42c5-94d2-517c74d2284c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:04.253434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-9bc5b74c-5df7-42c5-94d2-517c74d2284c None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:04.254674 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-9bc5b74c-5df7-42c5-94d2-517c74d2284c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:04.318223 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1164/2307] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:04.326446 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9629a5-1a1f-47cf-a736-31df0ba9d5e0 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:04.327459 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9629a5-1a1f-47cf-a736-31df0ba9d5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:04.327832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9629a5-1a1f-47cf-a736-31df0ba9d5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:04.347693 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-8f9629a5-1a1f-47cf-a736-31df0ba9d5e0 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:04.348912 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-8f9629a5-1a1f-47cf-a736-31df0ba9d5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:04.356757 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1144/2308] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:04.361711 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5b74c-5df7-42c5-94d2-517c74d2284c None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:04.371278 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5b74c-5df7-42c5-94d2-517c74d2284c None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:04.376540 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903977a-1e13-4373-bd92-b8e870c64132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:04.377179 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903977a-1e13-4373-bd92-b8e870c64132 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:04.377476 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903977a-1e13-4373-bd92-b8e870c64132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:04.377727 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903977a-1e13-4373-bd92-b8e870c64132 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:04.378412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903977a-1e13-4373-bd92-b8e870c64132 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:42:04.378713 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903977a-1e13-4373-bd92-b8e870c64132 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:04.380312 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903977a-1e13-4373-bd92-b8e870c64132 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:04.381790 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:04.397012 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1165/2309] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:42:04.398099 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9629a5-1a1f-47cf-a736-31df0ba9d5e0 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:04.407225 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9629a5-1a1f-47cf-a736-31df0ba9d5e0 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:04.411844 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d832de-49f8-4490-9862-a4cff03bc5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:04.412447 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d832de-49f8-4490-9862-a4cff03bc5ba admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:04.412730 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d832de-49f8-4490-9862-a4cff03bc5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:04.413035 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d832de-49f8-4490-9862-a4cff03bc5ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:04.413698 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d832de-49f8-4490-9862-a4cff03bc5ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:04.415635 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d832de-49f8-4490-9862-a4cff03bc5ba admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:04.443766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1145/2310] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:42:04 2022] POST /identity/v3/projects => generated 405 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:42:04.447941 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903977a-1e13-4373-bd92-b8e870c64132 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:04.457714 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903977a-1e13-4373-bd92-b8e870c64132 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:04.513422 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30297d71-fbe9-4f18-a250-15013e118cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:04.514042 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30297d71-fbe9-4f18-a250-15013e118cf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:04.514328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30297d71-fbe9-4f18-a250-15013e118cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:04.514608 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30297d71-fbe9-4f18-a250-15013e118cf4 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:04.515273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30297d71-fbe9-4f18-a250-15013e118cf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:04.517243 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30297d71-fbe9-4f18-a250-15013e118cf4 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:04.537697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1166/2311] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:42:04 2022] POST /identity/v3/users => generated 303 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:42:04.541797 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d832de-49f8-4490-9862-a4cff03bc5ba admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:04.552474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d832de-49f8-4490-9862-a4cff03bc5ba admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:04.557088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6ec23-2031-4d28-a817-296f5afefd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:04.557695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6ec23-2031-4d28-a817-296f5afefd3f admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:04.557976 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6ec23-2031-4d28-a817-296f5afefd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:04.558249 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6ec23-2031-4d28-a817-296f5afefd3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:04.558938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb6ec23-2031-4d28-a817-296f5afefd3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:04.560529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb6ec23-2031-4d28-a817-296f5afefd3f admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:04.570859 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1146/2312] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:42:04.574393 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30297d71-fbe9-4f18-a250-15013e118cf4 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:04.584513 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30297d71-fbe9-4f18-a250-15013e118cf4 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:04.590101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5fbe1-8798-490f-ba62-91c6ba002874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:04.590746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5fbe1-8798-490f-ba62-91c6ba002874 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:04.591262 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5fbe1-8798-490f-ba62-91c6ba002874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:04.591485 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5fbe1-8798-490f-ba62-91c6ba002874 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:04.592172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c5fbe1-8798-490f-ba62-91c6ba002874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:04.594152 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c5fbe1-8798-490f-ba62-91c6ba002874 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:04.607130 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1167/2313] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:42:04.612652 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6ec23-2031-4d28-a817-296f5afefd3f admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:04.622795 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6ec23-2031-4d28-a817-296f5afefd3f admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:04.628348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a9040-94eb-4c29-be90-aa0d0672d284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:04.629049 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a9040-94eb-4c29-be90-aa0d0672d284 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:04.629330 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a9040-94eb-4c29-be90-aa0d0672d284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:04.629603 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a9040-94eb-4c29-be90-aa0d0672d284 admin admin] PATH_INFO: `/v3/projects/ddc8b1f1f9fc4749b3fe34f66a8e0437/users/1e4bab78f24b4eb581cbee800783be43/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:04.647712 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a9040-94eb-4c29-be90-aa0d0672d284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddc8b1f1f9fc4749b3fe34f66a8e0437, user_id=1e4bab78f24b4eb581cbee800783be43, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:04.650637 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a9040-94eb-4c29-be90-aa0d0672d284 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:04.666609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1147/2314] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:42:04 2022] PUT /identity/v3/projects/ddc8b1f1f9fc4749b3fe34f66a8e0437/users/1e4bab78f24b4eb581cbee800783be43/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:04.678055 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f86b8a-e3ff-49c9-91db-d44b733f9295 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:04.678348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f86b8a-e3ff-49c9-91db-d44b733f9295 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:04.678633 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f86b8a-e3ff-49c9-91db-d44b733f9295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:04.719748 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-e1f86b8a-e3ff-49c9-91db-d44b733f9295 None None] MFA Rules not processed for user `1e4bab78f24b4eb581cbee800783be43`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:04.721125 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e1f86b8a-e3ff-49c9-91db-d44b733f9295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:04.804845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1168/2315] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:04 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:04.809877 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a9040-94eb-4c29-be90-aa0d0672d284 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:04.823922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:04.875997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a9040-94eb-4c29-be90-aa0d0672d284 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:04.880682 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c7376-cb92-4bd6-b0dc-a062562206f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:04.881326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c7376-cb92-4bd6-b0dc-a062562206f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:04.881611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c7376-cb92-4bd6-b0dc-a062562206f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:04.881886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c7376-cb92-4bd6-b0dc-a062562206f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:04.882541 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c7376-cb92-4bd6-b0dc-a062562206f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:04.884444 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c7376-cb92-4bd6-b0dc-a062562206f4 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:04.910041 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1148/2316] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:42:04 2022] POST /identity/v3/projects => generated 405 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:42:04.914365 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f86b8a-e3ff-49c9-91db-d44b733f9295 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:04.925547 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f86b8a-e3ff-49c9-91db-d44b733f9295 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:04.979482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28761b-d374-44e0-bd36-1e91b5f880c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:04.980385 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28761b-d374-44e0-bd36-1e91b5f880c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:04.980673 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28761b-d374-44e0-bd36-1e91b5f880c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:04.980983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28761b-d374-44e0-bd36-1e91b5f880c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:04.981634 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28761b-d374-44e0-bd36-1e91b5f880c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:04.984221 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28761b-d374-44e0-bd36-1e91b5f880c4 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:04.994712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:05.008191 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1169/2317] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:42:04 2022] POST /identity/v3/users => generated 309 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:42:05.012878 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c7376-cb92-4bd6-b0dc-a062562206f4 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:05.024483 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c7376-cb92-4bd6-b0dc-a062562206f4 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:05.030459 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c7709-0299-4112-a5f6-8b6dd3f79cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:05.031141 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c7709-0299-4112-a5f6-8b6dd3f79cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:05.031656 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c7709-0299-4112-a5f6-8b6dd3f79cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:05.031945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c7709-0299-4112-a5f6-8b6dd3f79cc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:05.032649 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c7709-0299-4112-a5f6-8b6dd3f79cc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:05.035173 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c7709-0299-4112-a5f6-8b6dd3f79cc8 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:05.048454 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1149/2318] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:42:05.052726 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28761b-d374-44e0-bd36-1e91b5f880c4 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:05.062845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28761b-d374-44e0-bd36-1e91b5f880c4 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:05.067464 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f7908-5920-406e-88ad-5832c761ce42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:05.068150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f7908-5920-406e-88ad-5832c761ce42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:05.068434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f7908-5920-406e-88ad-5832c761ce42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:05.068713 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f7908-5920-406e-88ad-5832c761ce42 admin admin] PATH_INFO: `/v3/projects/9626ee96c4fb4407ac44fdb5b034c985/users/546c8aa54ba34721a66eda875ce528a7/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:05.083273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f7908-5920-406e-88ad-5832c761ce42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9626ee96c4fb4407ac44fdb5b034c985, user_id=546c8aa54ba34721a66eda875ce528a7, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:05.085917 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f7908-5920-406e-88ad-5832c761ce42 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:05.100220 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1170/2319] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:42:05 2022] PUT /identity/v3/projects/9626ee96c4fb4407ac44fdb5b034c985/users/546c8aa54ba34721a66eda875ce528a7/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:05.104148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c7709-0299-4112-a5f6-8b6dd3f79cc8 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:05.114908 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c7709-0299-4112-a5f6-8b6dd3f79cc8 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:05.173087 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88a385-3d29-4e43-bb1a-c6a3ee60fde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:05.173712 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88a385-3d29-4e43-bb1a-c6a3ee60fde8 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:05.173991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88a385-3d29-4e43-bb1a-c6a3ee60fde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:05.174261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88a385-3d29-4e43-bb1a-c6a3ee60fde8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:05.175039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a88a385-3d29-4e43-bb1a-c6a3ee60fde8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:05.176631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a88a385-3d29-4e43-bb1a-c6a3ee60fde8 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:05.187301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1150/2320] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:42:05.191898 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f7908-5920-406e-88ad-5832c761ce42 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:05.201956 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f7908-5920-406e-88ad-5832c761ce42 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:05.206925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94607b5a-b69b-44ca-9642-15d95e2b2570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:05.207579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94607b5a-b69b-44ca-9642-15d95e2b2570 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:05.207884 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94607b5a-b69b-44ca-9642-15d95e2b2570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:05.208155 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94607b5a-b69b-44ca-9642-15d95e2b2570 admin admin] PATH_INFO: `/v3/projects/9626ee96c4fb4407ac44fdb5b034c985/users/546c8aa54ba34721a66eda875ce528a7/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:05.210085 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94607b5a-b69b-44ca-9642-15d95e2b2570 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9626ee96c4fb4407ac44fdb5b034c985, user_id=546c8aa54ba34721a66eda875ce528a7, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:05.212701 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94607b5a-b69b-44ca-9642-15d95e2b2570 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:05.224811 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1171/2321] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:42:05 2022] PUT /identity/v3/projects/9626ee96c4fb4407ac44fdb5b034c985/users/546c8aa54ba34721a66eda875ce528a7/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:42:05.235704 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d142ac-8b36-4027-b9a2-9bec1407e691 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:05.235997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d142ac-8b36-4027-b9a2-9bec1407e691 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:05.236272 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d142ac-8b36-4027-b9a2-9bec1407e691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:05.284796 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-43d142ac-8b36-4027-b9a2-9bec1407e691 None None] MFA Rules not processed for user `546c8aa54ba34721a66eda875ce528a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:05.286353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-43d142ac-8b36-4027-b9a2-9bec1407e691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:05.364179 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1151/2322] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:05 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:05.372872 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94607b5a-b69b-44ca-9642-15d95e2b2570 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:05.375741 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-94607b5a-b69b-44ca-9642-15d95e2b2570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:05.446906 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94607b5a-b69b-44ca-9642-15d95e2b2570 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:05.450742 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b4e0a-e7f5-4a48-a30a-17fecacb532a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:05.451367 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b4e0a-e7f5-4a48-a30a-17fecacb532a service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:05.451674 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b4e0a-e7f5-4a48-a30a-17fecacb532a service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:05.451954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b4e0a-e7f5-4a48-a30a-17fecacb532a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:05.460971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b4e0a-e7f5-4a48-a30a-17fecacb532a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:05.462422 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b4e0a-e7f5-4a48-a30a-17fecacb532a service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:05.472016 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1172/2323] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:42:05 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:42:05.603915 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d142ac-8b36-4027-b9a2-9bec1407e691 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:05.613717 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:05.665973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d142ac-8b36-4027-b9a2-9bec1407e691 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:05.670341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d143a-86cc-4d98-bced-8bc250339702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:05.670968 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d143a-86cc-4d98-bced-8bc250339702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:05.671245 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d143a-86cc-4d98-bced-8bc250339702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:05.671523 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d143a-86cc-4d98-bced-8bc250339702 admin admin] PATH_INFO: `/v3/users/1e4bab78f24b4eb581cbee800783be43` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:05.672651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d143a-86cc-4d98-bced-8bc250339702 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e4bab78f24b4eb581cbee800783be43)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:05.674589 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d143a-86cc-4d98-bced-8bc250339702 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:05.721347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-788d143a-86cc-4d98-bced-8bc250339702 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e4bab78f24b4eb581cbee800783be43'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:05.728429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-788d143a-86cc-4d98-bced-8bc250339702 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e4bab78f24b4eb581cbee800783be43'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:05.735819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-788d143a-86cc-4d98-bced-8bc250339702 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e4bab78f24b4eb581cbee800783be43'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:05.747449 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-788d143a-86cc-4d98-bced-8bc250339702 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e4bab78f24b4eb581cbee800783be43'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:05.748135 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-788d143a-86cc-4d98-bced-8bc250339702 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e4bab78f24b4eb581cbee800783be43'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:05.754206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1152/2324] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:42:05 2022] DELETE /identity/v3/users/1e4bab78f24b4eb581cbee800783be43 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:05.758749 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e44e1-00d3-4a23-89e5-6b50da4145cd None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:05.759041 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e44e1-00d3-4a23-89e5-6b50da4145cd None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:05.759318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e44e1-00d3-4a23-89e5-6b50da4145cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:05.777853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-dd9e44e1-00d3-4a23-89e5-6b50da4145cd None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:05.779065 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-dd9e44e1-00d3-4a23-89e5-6b50da4145cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:05.837711 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1173/2325] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:05.847568 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d143a-86cc-4d98-bced-8bc250339702 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:05.849130 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-788d143a-86cc-4d98-bced-8bc250339702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:05.907342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d143a-86cc-4d98-bced-8bc250339702 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:05.910975 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6f4f1-0702-4198-97f6-4b856485b6d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:05.911588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6f4f1-0702-4198-97f6-4b856485b6d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:05.911872 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6f4f1-0702-4198-97f6-4b856485b6d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:05.912147 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6f4f1-0702-4198-97f6-4b856485b6d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:05.922928 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6f4f1-0702-4198-97f6-4b856485b6d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:05.923943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6f4f1-0702-4198-97f6-4b856485b6d3 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:05.933537 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1153/2326] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:42:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:42:06.531698 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e44e1-00d3-4a23-89e5-6b50da4145cd None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:06.533194 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-dd9e44e1-00d3-4a23-89e5-6b50da4145cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:06.545558 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e44e1-00d3-4a23-89e5-6b50da4145cd None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:06.550087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dd012-655f-48c2-b744-9fa3f83e9cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:06.550728 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dd012-655f-48c2-b744-9fa3f83e9cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:06.551037 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dd012-655f-48c2-b744-9fa3f83e9cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:06.551310 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dd012-655f-48c2-b744-9fa3f83e9cbf admin admin] PATH_INFO: `/v3/projects/ddc8b1f1f9fc4749b3fe34f66a8e0437` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:06.552462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02dd012-655f-48c2-b744-9fa3f83e9cbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddc8b1f1f9fc4749b3fe34f66a8e0437)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:06.554411 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02dd012-655f-48c2-b744-9fa3f83e9cbf admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:06.615680 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a02dd012-655f-48c2-b744-9fa3f83e9cbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddc8b1f1f9fc4749b3fe34f66a8e0437', 'request_id': 'req-a02dd012-655f-48c2-b744-9fa3f83e9cbf'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:06.618757 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:06.625084 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a02dd012-655f-48c2-b744-9fa3f83e9cbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddc8b1f1f9fc4749b3fe34f66a8e0437', 'request_id': 'req-a02dd012-655f-48c2-b744-9fa3f83e9cbf'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:06.631567 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a02dd012-655f-48c2-b744-9fa3f83e9cbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddc8b1f1f9fc4749b3fe34f66a8e0437', 'request_id': 'req-a02dd012-655f-48c2-b744-9fa3f83e9cbf'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:06.643217 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a02dd012-655f-48c2-b744-9fa3f83e9cbf admin admin] The token cache is being invalidate because project ddc8b1f1f9fc4749b3fe34f66a8e0437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:42:06.643509 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a02dd012-655f-48c2-b744-9fa3f83e9cbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:06.644221 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-a02dd012-655f-48c2-b744-9fa3f83e9cbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:06.645969 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1174/2327] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:42:06 2022] DELETE /identity/v3/projects/ddc8b1f1f9fc4749b3fe34f66a8e0437 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:06.649681 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6f4f1-0702-4198-97f6-4b856485b6d3 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:06.659499 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6f4f1-0702-4198-97f6-4b856485b6d3 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:06.689425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:06.720380 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac995b-660d-447f-a831-53e526673446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:06.721077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac995b-660d-447f-a831-53e526673446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:06.721366 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac995b-660d-447f-a831-53e526673446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:06.721648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac995b-660d-447f-a831-53e526673446 admin admin] PATH_INFO: `/v3/users/546c8aa54ba34721a66eda875ce528a7` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:06.722828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac995b-660d-447f-a831-53e526673446 admin admin] RBAC: Authorizing `identity:delete_user(user_id=546c8aa54ba34721a66eda875ce528a7)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:06.725065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac995b-660d-447f-a831-53e526673446 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:06.773311 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-9bac995b-660d-447f-a831-53e526673446 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '546c8aa54ba34721a66eda875ce528a7'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:06.781816 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-9bac995b-660d-447f-a831-53e526673446 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '546c8aa54ba34721a66eda875ce528a7'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:06.789377 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-9bac995b-660d-447f-a831-53e526673446 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '546c8aa54ba34721a66eda875ce528a7'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:06.803645 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-9bac995b-660d-447f-a831-53e526673446 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '546c8aa54ba34721a66eda875ce528a7'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:06.804837 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-9bac995b-660d-447f-a831-53e526673446 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '546c8aa54ba34721a66eda875ce528a7'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:06.816001 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1154/2328] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:42:06 2022] DELETE /identity/v3/users/546c8aa54ba34721a66eda875ce528a7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:42:06.822228 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dd012-655f-48c2-b744-9fa3f83e9cbf admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:06.823709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-a02dd012-655f-48c2-b744-9fa3f83e9cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:06.890710 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dd012-655f-48c2-b744-9fa3f83e9cbf admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:06.941395 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b526a-6ccf-440c-a7f2-dc11de818b9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:06.942081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b526a-6ccf-440c-a7f2-dc11de818b9e service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:06.942420 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b526a-6ccf-440c-a7f2-dc11de818b9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:06.942725 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b526a-6ccf-440c-a7f2-dc11de818b9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:06.944803 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-524b526a-6ccf-440c-a7f2-dc11de818b9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:07.021113 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b526a-6ccf-440c-a7f2-dc11de818b9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:07.022643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b526a-6ccf-440c-a7f2-dc11de818b9e service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:07.035023 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1175/2329] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:42:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:42:07.477902 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac995b-660d-447f-a831-53e526673446 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:07.480296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9bac995b-660d-447f-a831-53e526673446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:07.631693 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac995b-660d-447f-a831-53e526673446 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:07.640614 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0dc943-63e6-4b5c-b3d8-17bc01740319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:07.641366 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0dc943-63e6-4b5c-b3d8-17bc01740319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:07.641721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0dc943-63e6-4b5c-b3d8-17bc01740319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:07.642093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0dc943-63e6-4b5c-b3d8-17bc01740319 admin admin] PATH_INFO: `/v3/projects/9626ee96c4fb4407ac44fdb5b034c985` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:07.643396 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0dc943-63e6-4b5c-b3d8-17bc01740319 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9626ee96c4fb4407ac44fdb5b034c985)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:07.655677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0dc943-63e6-4b5c-b3d8-17bc01740319 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:07.730177 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-df0dc943-63e6-4b5c-b3d8-17bc01740319 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9626ee96c4fb4407ac44fdb5b034c985', 'request_id': 'req-df0dc943-63e6-4b5c-b3d8-17bc01740319'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:07.733958 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:07.749947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-df0dc943-63e6-4b5c-b3d8-17bc01740319 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9626ee96c4fb4407ac44fdb5b034c985', 'request_id': 'req-df0dc943-63e6-4b5c-b3d8-17bc01740319'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:07.762473 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-df0dc943-63e6-4b5c-b3d8-17bc01740319 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9626ee96c4fb4407ac44fdb5b034c985', 'request_id': 'req-df0dc943-63e6-4b5c-b3d8-17bc01740319'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:07.782284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-df0dc943-63e6-4b5c-b3d8-17bc01740319 admin admin] The token cache is being invalidate because project 9626ee96c4fb4407ac44fdb5b034c985 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:42:07.782743 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-df0dc943-63e6-4b5c-b3d8-17bc01740319 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:07.783733 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-df0dc943-63e6-4b5c-b3d8-17bc01740319 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:07.785874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1155/2330] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:42:07 2022] DELETE /identity/v3/projects/9626ee96c4fb4407ac44fdb5b034c985 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:07.790715 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc51501-c535-4497-a387-3dbbca17f818 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:07.791029 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc51501-c535-4497-a387-3dbbca17f818 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:07.791268 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc51501-c535-4497-a387-3dbbca17f818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:07.812217 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-9bc51501-c535-4497-a387-3dbbca17f818 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:07.813644 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-9bc51501-c535-4497-a387-3dbbca17f818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:07.863334 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:07.893901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1176/2331] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:07.901872 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437259a-165c-45b1-acef-2b66743f0e46 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:07.902425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437259a-165c-45b1-acef-2b66743f0e46 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:07.902890 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437259a-165c-45b1-acef-2b66743f0e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:07.926648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-b437259a-165c-45b1-acef-2b66743f0e46 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:07.928215 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b437259a-165c-45b1-acef-2b66743f0e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:07.941166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1156/2332] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:42:07.945255 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc51501-c535-4497-a387-3dbbca17f818 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:07.961977 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc51501-c535-4497-a387-3dbbca17f818 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:07.973733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a43516-def9-496e-88b8-a326c612e0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:07.974374 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a43516-def9-496e-88b8-a326c612e0ae admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:07.974660 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a43516-def9-496e-88b8-a326c612e0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:07.974937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a43516-def9-496e-88b8-a326c612e0ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:07.975643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a43516-def9-496e-88b8-a326c612e0ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:42:07.975950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a43516-def9-496e-88b8-a326c612e0ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:07.978445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a43516-def9-496e-88b8-a326c612e0ae admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:07.994260 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1177/2333] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:42:07.995644 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437259a-165c-45b1-acef-2b66743f0e46 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:08.012086 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437259a-165c-45b1-acef-2b66743f0e46 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:08.020759 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf86fa-0b7c-4b58-bc9a-04c693fde4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:08.021554 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf86fa-0b7c-4b58-bc9a-04c693fde4ed admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:08.021907 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf86fa-0b7c-4b58-bc9a-04c693fde4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:08.022289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf86fa-0b7c-4b58-bc9a-04c693fde4ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:08.023051 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdf86fa-0b7c-4b58-bc9a-04c693fde4ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:08.025381 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdf86fa-0b7c-4b58-bc9a-04c693fde4ed admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:08.062063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1157/2334] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:42:07 2022] POST /identity/v3/projects => generated 365 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:42:08.066138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a43516-def9-496e-88b8-a326c612e0ae admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:08.077676 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a43516-def9-496e-88b8-a326c612e0ae admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:08.163795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e373b5-2c16-4089-bad1-de1321f07e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:08.164471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e373b5-2c16-4089-bad1-de1321f07e55 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:08.164731 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e373b5-2c16-4089-bad1-de1321f07e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:08.165068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e373b5-2c16-4089-bad1-de1321f07e55 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:08.165712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e373b5-2c16-4089-bad1-de1321f07e55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:08.168628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e373b5-2c16-4089-bad1-de1321f07e55 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:08.194451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1178/2335] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:42:08 2022] POST /identity/v3/users => generated 283 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:42:08.198270 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf86fa-0b7c-4b58-bc9a-04c693fde4ed admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:08.211355 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf86fa-0b7c-4b58-bc9a-04c693fde4ed admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:08.217491 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427fd4fb-234a-41cb-b01b-9e566614e4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:08.218693 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fd4fb-234a-41cb-b01b-9e566614e4bb admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:08.219408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fd4fb-234a-41cb-b01b-9e566614e4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:08.219755 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fd4fb-234a-41cb-b01b-9e566614e4bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:08.220533 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427fd4fb-234a-41cb-b01b-9e566614e4bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:08.222409 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427fd4fb-234a-41cb-b01b-9e566614e4bb admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:08.236957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1158/2336] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:42:08.240872 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e373b5-2c16-4089-bad1-de1321f07e55 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:08.255020 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e373b5-2c16-4089-bad1-de1321f07e55 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:08.262021 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4cbd70-5018-4a1a-b762-f77fd94888f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:08.263070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cbd70-5018-4a1a-b762-f77fd94888f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:08.263629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cbd70-5018-4a1a-b762-f77fd94888f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:08.264516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cbd70-5018-4a1a-b762-f77fd94888f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:08.265503 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4cbd70-5018-4a1a-b762-f77fd94888f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:08.268257 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4cbd70-5018-4a1a-b762-f77fd94888f7 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:08.283766 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1179/2337] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:42:08.289432 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427fd4fb-234a-41cb-b01b-9e566614e4bb admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:08.305868 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427fd4fb-234a-41cb-b01b-9e566614e4bb admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:08.312796 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51ddf6-8b70-42bb-a9a6-480fa48dcdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:08.313556 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51ddf6-8b70-42bb-a9a6-480fa48dcdaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:08.313888 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51ddf6-8b70-42bb-a9a6-480fa48dcdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:08.314168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51ddf6-8b70-42bb-a9a6-480fa48dcdaf admin admin] PATH_INFO: `/v3/projects/28a5da59467e4798b54e8ba921f92fbd/users/8100a83f751d42248c7b61309d51615e/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:08.334663 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51ddf6-8b70-42bb-a9a6-480fa48dcdaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=28a5da59467e4798b54e8ba921f92fbd, user_id=8100a83f751d42248c7b61309d51615e, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:08.337642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51ddf6-8b70-42bb-a9a6-480fa48dcdaf admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:08.360927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1159/2338] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:42:08 2022] PUT /identity/v3/projects/28a5da59467e4798b54e8ba921f92fbd/users/8100a83f751d42248c7b61309d51615e/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:08.370770 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb840c-eedc-410d-b738-18b184050146 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:08.371334 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb840c-eedc-410d-b738-18b184050146 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:08.371612 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb840c-eedc-410d-b738-18b184050146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:08.418974 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-f7fb840c-eedc-410d-b738-18b184050146 None None] MFA Rules not processed for user `8100a83f751d42248c7b61309d51615e`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:08.420796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f7fb840c-eedc-410d-b738-18b184050146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:08.535162 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1180/2339] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:08 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:08.535722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51ddf6-8b70-42bb-a9a6-480fa48dcdaf admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:08.636472 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51ddf6-8b70-42bb-a9a6-480fa48dcdaf admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:08.644294 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48198916-bc48-4f08-8d51-a16c647cea91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:08.645648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48198916-bc48-4f08-8d51-a16c647cea91 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:08.646188 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48198916-bc48-4f08-8d51-a16c647cea91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:08.648263 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48198916-bc48-4f08-8d51-a16c647cea91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:08.649234 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48198916-bc48-4f08-8d51-a16c647cea91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:08.651730 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48198916-bc48-4f08-8d51-a16c647cea91 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:08.704135 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb840c-eedc-410d-b738-18b184050146 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:08.713353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1160/2340] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:42:08 2022] POST /identity/v3/projects => generated 363 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:42:08.718783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb840c-eedc-410d-b738-18b184050146 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:08.785026 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:08.821873 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591193c9-8bc6-470d-8cca-551b4d40f0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:08.822523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591193c9-8bc6-470d-8cca-551b4d40f0a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:08.822806 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591193c9-8bc6-470d-8cca-551b4d40f0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:08.823083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591193c9-8bc6-470d-8cca-551b4d40f0a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:08.823755 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591193c9-8bc6-470d-8cca-551b4d40f0a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:08.825848 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591193c9-8bc6-470d-8cca-551b4d40f0a0 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:08.858544 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1181/2341] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:42:08 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:42:08.861505 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48198916-bc48-4f08-8d51-a16c647cea91 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:08.872711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48198916-bc48-4f08-8d51-a16c647cea91 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:08.877711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2607e-897e-4a21-8392-185bc10f1feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:08.878442 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2607e-897e-4a21-8392-185bc10f1feb admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:08.878654 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2607e-897e-4a21-8392-185bc10f1feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:08.879074 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2607e-897e-4a21-8392-185bc10f1feb admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:08.879794 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2607e-897e-4a21-8392-185bc10f1feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:08.881637 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2607e-897e-4a21-8392-185bc10f1feb admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:08.898016 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1161/2342] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:42:08.898967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591193c9-8bc6-470d-8cca-551b4d40f0a0 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:08.913813 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591193c9-8bc6-470d-8cca-551b4d40f0a0 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:08.919626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e6eac-7194-4d45-8343-d7ca7fe7ca92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:08.920362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e6eac-7194-4d45-8343-d7ca7fe7ca92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:08.920653 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e6eac-7194-4d45-8343-d7ca7fe7ca92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:08.920963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e6eac-7194-4d45-8343-d7ca7fe7ca92 admin admin] PATH_INFO: `/v3/projects/2c644ab597604f7b80373782e2a4f904/users/bede8d408b5d472a8c39c46e4ff3e15d/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:08.940223 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e6eac-7194-4d45-8343-d7ca7fe7ca92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c644ab597604f7b80373782e2a4f904, user_id=bede8d408b5d472a8c39c46e4ff3e15d, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:08.943117 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e6eac-7194-4d45-8343-d7ca7fe7ca92 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:08.960079 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1182/2343] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:42:08 2022] PUT /identity/v3/projects/2c644ab597604f7b80373782e2a4f904/users/bede8d408b5d472a8c39c46e4ff3e15d/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:42:08.964282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2607e-897e-4a21-8392-185bc10f1feb admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:08.978271 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2607e-897e-4a21-8392-185bc10f1feb admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:09.061024 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:09.075148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0284e3-b6b5-4075-8a13-a1ed399430bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:09.076246 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0284e3-b6b5-4075-8a13-a1ed399430bb admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:09.077227 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0284e3-b6b5-4075-8a13-a1ed399430bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:09.077605 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0284e3-b6b5-4075-8a13-a1ed399430bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:09.078542 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0284e3-b6b5-4075-8a13-a1ed399430bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:09.080363 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0284e3-b6b5-4075-8a13-a1ed399430bb admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:09.096792 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1162/2344] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:42:09.098501 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e6eac-7194-4d45-8343-d7ca7fe7ca92 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:09.113405 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e6eac-7194-4d45-8343-d7ca7fe7ca92 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:09.120075 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a8f79-4ae6-4205-8597-223735aed431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:09.120782 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a8f79-4ae6-4205-8597-223735aed431 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:09.121095 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a8f79-4ae6-4205-8597-223735aed431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:09.121377 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a8f79-4ae6-4205-8597-223735aed431 admin admin] PATH_INFO: `/v3/projects/2c644ab597604f7b80373782e2a4f904/users/bede8d408b5d472a8c39c46e4ff3e15d/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:09.126844 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a8f79-4ae6-4205-8597-223735aed431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c644ab597604f7b80373782e2a4f904, user_id=bede8d408b5d472a8c39c46e4ff3e15d, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:09.129700 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a8f79-4ae6-4205-8597-223735aed431 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:09.155419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1183/2345] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:42:09 2022] PUT /identity/v3/projects/2c644ab597604f7b80373782e2a4f904/users/bede8d408b5d472a8c39c46e4ff3e15d/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:09.155648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182af16-8c42-4f7e-8548-115b94399205 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:09.155648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182af16-8c42-4f7e-8548-115b94399205 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:09.155648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182af16-8c42-4f7e-8548-115b94399205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:09.200086 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-2182af16-8c42-4f7e-8548-115b94399205 None None] MFA Rules not processed for user `bede8d408b5d472a8c39c46e4ff3e15d`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:09.201942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-2182af16-8c42-4f7e-8548-115b94399205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:09.309766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1163/2346] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:09 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:42:09.321581 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a8f79-4ae6-4205-8597-223735aed431 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:09.323896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d03a8f79-4ae6-4205-8597-223735aed431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:09.434786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a8f79-4ae6-4205-8597-223735aed431 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:09.441252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de273d-d9a0-4b3c-a9de-c3a192ece7da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:09.442002 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de273d-d9a0-4b3c-a9de-c3a192ece7da service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:09.442460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de273d-d9a0-4b3c-a9de-c3a192ece7da service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:09.442709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de273d-d9a0-4b3c-a9de-c3a192ece7da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:09.493520 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de273d-d9a0-4b3c-a9de-c3a192ece7da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:09.495469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de273d-d9a0-4b3c-a9de-c3a192ece7da service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:09.512018 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1184/2347] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:42:09 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:42:09.613343 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182af16-8c42-4f7e-8548-115b94399205 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:09.625108 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182af16-8c42-4f7e-8548-115b94399205 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:09.630793 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67867154-4086-45ed-9837-7690544ca1cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:09.632590 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67867154-4086-45ed-9837-7690544ca1cd service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:09.632590 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67867154-4086-45ed-9837-7690544ca1cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:09.632995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67867154-4086-45ed-9837-7690544ca1cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:09.725590 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67867154-4086-45ed-9837-7690544ca1cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:09.727491 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67867154-4086-45ed-9837-7690544ca1cd service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:09.742550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1164/2348] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:42:09 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:42:10.110764 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8e348-a39d-4938-a2fb-b0d7e35a2c27 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:10.112324 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8e348-a39d-4938-a2fb-b0d7e35a2c27 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:10.112775 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8e348-a39d-4938-a2fb-b0d7e35a2c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:10.138435 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-e6c8e348-a39d-4938-a2fb-b0d7e35a2c27 None None] MFA Rules not processed for user `07d6bffc69674103b4c6540598a780b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:10.141112 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e6c8e348-a39d-4938-a2fb-b0d7e35a2c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:10.231452 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1185/2349] 10.222.0.74 () {58 vars in 1009 bytes} [Wed Jul 27 16:42:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:10.248225 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67867154-4086-45ed-9837-7690544ca1cd service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:10.250339 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-67867154-4086-45ed-9837-7690544ca1cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:10.338568 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67867154-4086-45ed-9837-7690544ca1cd service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:10.345031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ac7a0-2460-4401-9788-0a37a49cb3b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb45ded9f75e4fa2973b564f83bdfaf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:10.345430 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ac7a0-2460-4401-9788-0a37a49cb3b2 service swift] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:10.345781 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ac7a0-2460-4401-9788-0a37a49cb3b2 service swift] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:10.346128 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ac7a0-2460-4401-9788-0a37a49cb3b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:10.355321 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ac7a0-2460-4401-9788-0a37a49cb3b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:10.356975 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ac7a0-2460-4401-9788-0a37a49cb3b2 service swift] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:10.362388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:10.373432 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1165/2350] 10.222.0.74 () {60 vars in 1384 bytes} [Wed Jul 27 16:42:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 16:42:20.222719 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=76,keystone:INSERT=5 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:20.714036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8e348-a39d-4938-a2fb-b0d7e35a2c27 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:20.780011 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8e348-a39d-4938-a2fb-b0d7e35a2c27 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:20.784593 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fe8c5-a35e-4079-ae95-79a6c4436fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:20.785272 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fe8c5-a35e-4079-ae95-79a6c4436fd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:20.785559 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fe8c5-a35e-4079-ae95-79a6c4436fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:20.785842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fe8c5-a35e-4079-ae95-79a6c4436fd7 admin admin] PATH_INFO: `/v3/users/8100a83f751d42248c7b61309d51615e` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:20.787148 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5fe8c5-a35e-4079-ae95-79a6c4436fd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8100a83f751d42248c7b61309d51615e)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:20.789111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5fe8c5-a35e-4079-ae95-79a6c4436fd7 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:20.838885 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-9c5fe8c5-a35e-4079-ae95-79a6c4436fd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8100a83f751d42248c7b61309d51615e'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:20.850491 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-9c5fe8c5-a35e-4079-ae95-79a6c4436fd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8100a83f751d42248c7b61309d51615e'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:20.867023 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-9c5fe8c5-a35e-4079-ae95-79a6c4436fd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8100a83f751d42248c7b61309d51615e'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:20.874416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-9c5fe8c5-a35e-4079-ae95-79a6c4436fd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8100a83f751d42248c7b61309d51615e'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:20.875160 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-9c5fe8c5-a35e-4079-ae95-79a6c4436fd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8100a83f751d42248c7b61309d51615e'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:20.881279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1186/2351] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:42:20 2022] DELETE /identity/v3/users/8100a83f751d42248c7b61309d51615e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:20.886173 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deaab99-2536-43a2-b78c-ef5496290fd7 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:20.886697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deaab99-2536-43a2-b78c-ef5496290fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:20.886786 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deaab99-2536-43a2-b78c-ef5496290fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:20.908781 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-0deaab99-2536-43a2-b78c-ef5496290fd7 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:20.910041 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0deaab99-2536-43a2-b78c-ef5496290fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:20.971626 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1166/2352] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:20.981312 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fe8c5-a35e-4079-ae95-79a6c4436fd7 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:20.982824 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-9c5fe8c5-a35e-4079-ae95-79a6c4436fd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:21.041020 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fe8c5-a35e-4079-ae95-79a6c4436fd7 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:21.044713 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ce7df-c3dc-4a05-aee3-e54154219580 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:21.045391 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ce7df-c3dc-4a05-aee3-e54154219580 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:21.045672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ce7df-c3dc-4a05-aee3-e54154219580 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:21.045949 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ce7df-c3dc-4a05-aee3-e54154219580 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:21.055042 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ce7df-c3dc-4a05-aee3-e54154219580 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:21.056472 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ce7df-c3dc-4a05-aee3-e54154219580 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:21.066449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1187/2353] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:42:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:42:21.343856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deaab99-2536-43a2-b78c-ef5496290fd7 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:21.345627 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0deaab99-2536-43a2-b78c-ef5496290fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:21.360682 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deaab99-2536-43a2-b78c-ef5496290fd7 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:21.366727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de08b4-eade-46df-b2ca-f969679ab9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:21.367327 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de08b4-eade-46df-b2ca-f969679ab9a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:21.367659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de08b4-eade-46df-b2ca-f969679ab9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:21.367896 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de08b4-eade-46df-b2ca-f969679ab9a9 admin admin] PATH_INFO: `/v3/projects/28a5da59467e4798b54e8ba921f92fbd` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:21.369058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de08b4-eade-46df-b2ca-f969679ab9a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28a5da59467e4798b54e8ba921f92fbd)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:21.371073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de08b4-eade-46df-b2ca-f969679ab9a9 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:21.417277 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-f6de08b4-eade-46df-b2ca-f969679ab9a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28a5da59467e4798b54e8ba921f92fbd', 'request_id': 'req-f6de08b4-eade-46df-b2ca-f969679ab9a9'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:21.422709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-f6de08b4-eade-46df-b2ca-f969679ab9a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28a5da59467e4798b54e8ba921f92fbd', 'request_id': 'req-f6de08b4-eade-46df-b2ca-f969679ab9a9'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:21.430041 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-f6de08b4-eade-46df-b2ca-f969679ab9a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28a5da59467e4798b54e8ba921f92fbd', 'request_id': 'req-f6de08b4-eade-46df-b2ca-f969679ab9a9'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:21.441693 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-f6de08b4-eade-46df-b2ca-f969679ab9a9 admin admin] The token cache is being invalidate because project 28a5da59467e4798b54e8ba921f92fbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:42:21.441979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-f6de08b4-eade-46df-b2ca-f969679ab9a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:21.442604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-f6de08b4-eade-46df-b2ca-f969679ab9a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:21.444092 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1167/2354] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:42:21 2022] DELETE /identity/v3/projects/28a5da59467e4798b54e8ba921f92fbd => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:21.448126 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ce7df-c3dc-4a05-aee3-e54154219580 service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:21.458861 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ce7df-c3dc-4a05-aee3-e54154219580 service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:21.477215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:21.518409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ab3cf-9be9-4121-b87a-00625774ac8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:21.519060 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ab3cf-9be9-4121-b87a-00625774ac8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:21.519373 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ab3cf-9be9-4121-b87a-00625774ac8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:21.519653 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ab3cf-9be9-4121-b87a-00625774ac8e admin admin] PATH_INFO: `/v3/users/bede8d408b5d472a8c39c46e4ff3e15d` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:21.520838 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ab3cf-9be9-4121-b87a-00625774ac8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bede8d408b5d472a8c39c46e4ff3e15d)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:21.523155 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ab3cf-9be9-4121-b87a-00625774ac8e admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:21.568715 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-7b2ab3cf-9be9-4121-b87a-00625774ac8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bede8d408b5d472a8c39c46e4ff3e15d'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:21.575956 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-7b2ab3cf-9be9-4121-b87a-00625774ac8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bede8d408b5d472a8c39c46e4ff3e15d'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:21.588674 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-7b2ab3cf-9be9-4121-b87a-00625774ac8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bede8d408b5d472a8c39c46e4ff3e15d'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:21.595856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-7b2ab3cf-9be9-4121-b87a-00625774ac8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bede8d408b5d472a8c39c46e4ff3e15d'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:21.596503 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-7b2ab3cf-9be9-4121-b87a-00625774ac8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bede8d408b5d472a8c39c46e4ff3e15d'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:21.602414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1188/2355] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:42:21 2022] DELETE /identity/v3/users/bede8d408b5d472a8c39c46e4ff3e15d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:21.868650 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de08b4-eade-46df-b2ca-f969679ab9a9 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:21.870225 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-f6de08b4-eade-46df-b2ca-f969679ab9a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:21.887909 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de08b4-eade-46df-b2ca-f969679ab9a9 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:21.946530 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2edfd-1cd3-4d91-850e-b0096ee3d88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:21.947288 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2edfd-1cd3-4d91-850e-b0096ee3d88e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:21.947642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2edfd-1cd3-4d91-850e-b0096ee3d88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:21.948012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2edfd-1cd3-4d91-850e-b0096ee3d88e admin admin] PATH_INFO: `/v3/projects/2c644ab597604f7b80373782e2a4f904` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:21.949427 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd2edfd-1cd3-4d91-850e-b0096ee3d88e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c644ab597604f7b80373782e2a4f904)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:21.951880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd2edfd-1cd3-4d91-850e-b0096ee3d88e admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:21.983835 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:22.003070 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6fd2edfd-1cd3-4d91-850e-b0096ee3d88e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c644ab597604f7b80373782e2a4f904', 'request_id': 'req-6fd2edfd-1cd3-4d91-850e-b0096ee3d88e'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:22.008835 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6fd2edfd-1cd3-4d91-850e-b0096ee3d88e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c644ab597604f7b80373782e2a4f904', 'request_id': 'req-6fd2edfd-1cd3-4d91-850e-b0096ee3d88e'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:22.017055 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6fd2edfd-1cd3-4d91-850e-b0096ee3d88e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c644ab597604f7b80373782e2a4f904', 'request_id': 'req-6fd2edfd-1cd3-4d91-850e-b0096ee3d88e'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:22.028552 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6fd2edfd-1cd3-4d91-850e-b0096ee3d88e admin admin] The token cache is being invalidate because project 2c644ab597604f7b80373782e2a4f904 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:42:22.028844 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6fd2edfd-1cd3-4d91-850e-b0096ee3d88e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:22.030096 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6fd2edfd-1cd3-4d91-850e-b0096ee3d88e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:22.031615 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1168/2356] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:42:21 2022] DELETE /identity/v3/projects/2c644ab597604f7b80373782e2a4f904 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:22.036451 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5162620-6e7e-4550-a95f-0018f27b0ad0 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:22.036744 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5162620-6e7e-4550-a95f-0018f27b0ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:22.037060 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5162620-6e7e-4550-a95f-0018f27b0ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:22.057625 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-a5162620-6e7e-4550-a95f-0018f27b0ad0 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:22.058847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-a5162620-6e7e-4550-a95f-0018f27b0ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:22.119733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1189/2357] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:22.128725 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d6c35-6734-437a-9b96-ddcf7a20b031 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:22.129046 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d6c35-6734-437a-9b96-ddcf7a20b031 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:22.129327 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d6c35-6734-437a-9b96-ddcf7a20b031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:22.148708 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-193d6c35-6734-437a-9b96-ddcf7a20b031 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:22.149939 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-193d6c35-6734-437a-9b96-ddcf7a20b031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:22.158724 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1169/2358] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:22.162949 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5162620-6e7e-4550-a95f-0018f27b0ad0 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:22.173661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5162620-6e7e-4550-a95f-0018f27b0ad0 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:22.178257 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00bc3c-7560-44a4-b317-c9d4228d59a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:22.178873 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00bc3c-7560-44a4-b317-c9d4228d59a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:22.179152 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00bc3c-7560-44a4-b317-c9d4228d59a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:22.179454 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00bc3c-7560-44a4-b317-c9d4228d59a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:22.180152 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00bc3c-7560-44a4-b317-c9d4228d59a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:42:22.180458 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00bc3c-7560-44a4-b317-c9d4228d59a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:22.182083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00bc3c-7560-44a4-b317-c9d4228d59a6 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:22.193507 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1190/2359] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:42:22.197745 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d6c35-6734-437a-9b96-ddcf7a20b031 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:22.206915 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d6c35-6734-437a-9b96-ddcf7a20b031 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:22.211601 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d077cb-1c28-41cf-93de-63f274d4e036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:22.212201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d077cb-1c28-41cf-93de-63f274d4e036 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:22.212485 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d077cb-1c28-41cf-93de-63f274d4e036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:22.212760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d077cb-1c28-41cf-93de-63f274d4e036 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:22.213469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d077cb-1c28-41cf-93de-63f274d4e036 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:22.215369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d077cb-1c28-41cf-93de-63f274d4e036 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:22.243748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1170/2360] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:42:22 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:42:22.247791 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00bc3c-7560-44a4-b317-c9d4228d59a6 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:22.257665 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00bc3c-7560-44a4-b317-c9d4228d59a6 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:22.282711 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:22.314051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86dc90b-2cda-4328-a99f-cd8b332623d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:22.314658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86dc90b-2cda-4328-a99f-cd8b332623d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:22.314936 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86dc90b-2cda-4328-a99f-cd8b332623d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:22.315207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86dc90b-2cda-4328-a99f-cd8b332623d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:22.315851 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86dc90b-2cda-4328-a99f-cd8b332623d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:22.317811 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86dc90b-2cda-4328-a99f-cd8b332623d0 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:22.337621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1191/2361] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:42:22 2022] POST /identity/v3/users => generated 294 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:42:22.341673 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d077cb-1c28-41cf-93de-63f274d4e036 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:22.352554 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d077cb-1c28-41cf-93de-63f274d4e036 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:22.357178 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c376c5e-0d7a-4b78-8df3-ad849c21dcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:22.358450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c376c5e-0d7a-4b78-8df3-ad849c21dcc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:22.358727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c376c5e-0d7a-4b78-8df3-ad849c21dcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:22.359001 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c376c5e-0d7a-4b78-8df3-ad849c21dcc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:22.359681 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c376c5e-0d7a-4b78-8df3-ad849c21dcc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:22.361308 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c376c5e-0d7a-4b78-8df3-ad849c21dcc2 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:22.370912 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1171/2362] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:42:22.374622 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86dc90b-2cda-4328-a99f-cd8b332623d0 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:22.384504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86dc90b-2cda-4328-a99f-cd8b332623d0 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:22.390823 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4101e-b7ac-444d-aa9a-7ccb89432c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:22.391428 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4101e-b7ac-444d-aa9a-7ccb89432c50 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:22.391711 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4101e-b7ac-444d-aa9a-7ccb89432c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:22.391986 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4101e-b7ac-444d-aa9a-7ccb89432c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:22.392674 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4101e-b7ac-444d-aa9a-7ccb89432c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:22.394374 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4101e-b7ac-444d-aa9a-7ccb89432c50 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:22.405381 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1192/2363] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:42:22.409229 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c376c5e-0d7a-4b78-8df3-ad849c21dcc2 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:22.419020 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c376c5e-0d7a-4b78-8df3-ad849c21dcc2 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:22.423575 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b8a8b-7993-412c-86ed-9a38169ef48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:22.424248 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b8a8b-7993-412c-86ed-9a38169ef48f admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:22.424528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b8a8b-7993-412c-86ed-9a38169ef48f admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:22.424800 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b8a8b-7993-412c-86ed-9a38169ef48f admin admin] PATH_INFO: `/v3/projects/06e8dbf44d8240a283ed46b979158f59/users/b46c0aff591b4e27892db2a266a81c85/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:22.438881 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b8a8b-7993-412c-86ed-9a38169ef48f admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e8dbf44d8240a283ed46b979158f59, user_id=b46c0aff591b4e27892db2a266a81c85, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:22.441550 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b8a8b-7993-412c-86ed-9a38169ef48f admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:22.454098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1172/2364] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:42:22 2022] PUT /identity/v3/projects/06e8dbf44d8240a283ed46b979158f59/users/b46c0aff591b4e27892db2a266a81c85/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:42:22.464372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30aab22-bcf8-4b7b-a3e0-5a82d9914602 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:22.464857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30aab22-bcf8-4b7b-a3e0-5a82d9914602 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:22.465034 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30aab22-bcf8-4b7b-a3e0-5a82d9914602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:22.492561 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-a30aab22-bcf8-4b7b-a3e0-5a82d9914602 None None] MFA Rules not processed for user `b46c0aff591b4e27892db2a266a81c85`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:22.493796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-a30aab22-bcf8-4b7b-a3e0-5a82d9914602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:22.558065 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1193/2365] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:22.563113 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b8a8b-7993-412c-86ed-9a38169ef48f admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:22.624504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b8a8b-7993-412c-86ed-9a38169ef48f admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:22.629148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc2a36-4abf-4565-9a42-039197d1b49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:22.629756 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc2a36-4abf-4565-9a42-039197d1b49a admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:22.630034 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc2a36-4abf-4565-9a42-039197d1b49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:22.630305 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc2a36-4abf-4565-9a42-039197d1b49a admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:22.630957 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc2a36-4abf-4565-9a42-039197d1b49a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:22.632869 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc2a36-4abf-4565-9a42-039197d1b49a admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:22.659555 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1173/2366] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:42:22 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:42:22.663809 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30aab22-bcf8-4b7b-a3e0-5a82d9914602 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:22.674595 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30aab22-bcf8-4b7b-a3e0-5a82d9914602 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:22.736458 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7c584-3380-4525-a0d2-cfe1113af5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:22.737197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7c584-3380-4525-a0d2-cfe1113af5df admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:22.737487 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7c584-3380-4525-a0d2-cfe1113af5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:22.737765 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7c584-3380-4525-a0d2-cfe1113af5df admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:22.738441 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e7c584-3380-4525-a0d2-cfe1113af5df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:22.740551 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e7c584-3380-4525-a0d2-cfe1113af5df admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:22.762991 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1194/2367] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:42:22 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:42:22.767483 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc2a36-4abf-4565-9a42-039197d1b49a admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:22.778287 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc2a36-4abf-4565-9a42-039197d1b49a admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:22.783340 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ea7f3-b226-42b6-aaac-6c4452156320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:22.783972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ea7f3-b226-42b6-aaac-6c4452156320 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:22.784255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ea7f3-b226-42b6-aaac-6c4452156320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:22.784531 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ea7f3-b226-42b6-aaac-6c4452156320 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:22.785475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ea7f3-b226-42b6-aaac-6c4452156320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:22.787112 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ea7f3-b226-42b6-aaac-6c4452156320 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:22.798484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1174/2368] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:42:22.802480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7c584-3380-4525-a0d2-cfe1113af5df admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:22.814555 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7c584-3380-4525-a0d2-cfe1113af5df admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:22.820648 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4498444f-7c38-406f-921b-d341f55008a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:22.821473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498444f-7c38-406f-921b-d341f55008a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:22.821891 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498444f-7c38-406f-921b-d341f55008a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:22.822268 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498444f-7c38-406f-921b-d341f55008a2 admin admin] PATH_INFO: `/v3/projects/abdb1ffa71ac4a51a266f38698b6b366/users/f6c22d0bb80d4d319b82358ec7a8b8ad/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:22.841064 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4498444f-7c38-406f-921b-d341f55008a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abdb1ffa71ac4a51a266f38698b6b366, user_id=f6c22d0bb80d4d319b82358ec7a8b8ad, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:22.843980 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4498444f-7c38-406f-921b-d341f55008a2 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:22.857605 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1195/2369] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:42:22 2022] PUT /identity/v3/projects/abdb1ffa71ac4a51a266f38698b6b366/users/f6c22d0bb80d4d319b82358ec7a8b8ad/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:42:22.861667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ea7f3-b226-42b6-aaac-6c4452156320 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:22.871997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ea7f3-b226-42b6-aaac-6c4452156320 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:22.893545 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:22.931936 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67695657-a2e1-45cf-a33f-468bc2490d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:22.932553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67695657-a2e1-45cf-a33f-468bc2490d52 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:22.932837 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67695657-a2e1-45cf-a33f-468bc2490d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:22.933144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67695657-a2e1-45cf-a33f-468bc2490d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:22.933836 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67695657-a2e1-45cf-a33f-468bc2490d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:22.935446 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67695657-a2e1-45cf-a33f-468bc2490d52 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:22.945529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1175/2370] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:42:22.949659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4498444f-7c38-406f-921b-d341f55008a2 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:22.955719 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:22.961800 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4498444f-7c38-406f-921b-d341f55008a2 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:22.966797 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af24d2e-1485-4d8c-aa12-2c101e0e0f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:22.967482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af24d2e-1485-4d8c-aa12-2c101e0e0f28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:22.967762 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af24d2e-1485-4d8c-aa12-2c101e0e0f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:22.968036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af24d2e-1485-4d8c-aa12-2c101e0e0f28 admin admin] PATH_INFO: `/v3/projects/abdb1ffa71ac4a51a266f38698b6b366/users/f6c22d0bb80d4d319b82358ec7a8b8ad/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:22.970268 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af24d2e-1485-4d8c-aa12-2c101e0e0f28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abdb1ffa71ac4a51a266f38698b6b366, user_id=f6c22d0bb80d4d319b82358ec7a8b8ad, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:22.972917 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af24d2e-1485-4d8c-aa12-2c101e0e0f28 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:22.983085 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1196/2371] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:42:22 2022] PUT /identity/v3/projects/abdb1ffa71ac4a51a266f38698b6b366/users/f6c22d0bb80d4d319b82358ec7a8b8ad/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:22.993133 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd1785-2f7b-4f76-a109-765faeb491c2 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:22.993426 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd1785-2f7b-4f76-a109-765faeb491c2 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:22.993708 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd1785-2f7b-4f76-a109-765faeb491c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:23.023183 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-55dd1785-2f7b-4f76-a109-765faeb491c2 None None] MFA Rules not processed for user `f6c22d0bb80d4d319b82358ec7a8b8ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:23.024394 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-55dd1785-2f7b-4f76-a109-765faeb491c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:23.097794 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1176/2372] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:23.111969 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af24d2e-1485-4d8c-aa12-2c101e0e0f28 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:23.113862 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-5af24d2e-1485-4d8c-aa12-2c101e0e0f28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:23.200746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af24d2e-1485-4d8c-aa12-2c101e0e0f28 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:23.204499 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b552710-4aaa-4360-be89-9c3332b16041 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:23.205418 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b552710-4aaa-4360-be89-9c3332b16041 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:23.205695 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b552710-4aaa-4360-be89-9c3332b16041 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:23.206364 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b552710-4aaa-4360-be89-9c3332b16041 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:23.216790 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b552710-4aaa-4360-be89-9c3332b16041 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:23.219557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b552710-4aaa-4360-be89-9c3332b16041 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:23.230135 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1197/2373] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:42:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:42:23.294036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd1785-2f7b-4f76-a109-765faeb491c2 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:23.355666 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd1785-2f7b-4f76-a109-765faeb491c2 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:23.360680 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1791b-0ddb-4fa3-ba8e-7a3fcac77f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:23.361442 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1791b-0ddb-4fa3-ba8e-7a3fcac77f3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:23.361727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1791b-0ddb-4fa3-ba8e-7a3fcac77f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:23.362004 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1791b-0ddb-4fa3-ba8e-7a3fcac77f3c admin admin] PATH_INFO: `/v3/users/b46c0aff591b4e27892db2a266a81c85` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:23.363264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1791b-0ddb-4fa3-ba8e-7a3fcac77f3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b46c0aff591b4e27892db2a266a81c85)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:23.365223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1791b-0ddb-4fa3-ba8e-7a3fcac77f3c admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:23.413456 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a6a1791b-0ddb-4fa3-ba8e-7a3fcac77f3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b46c0aff591b4e27892db2a266a81c85'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:23.415995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:23.423777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a6a1791b-0ddb-4fa3-ba8e-7a3fcac77f3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b46c0aff591b4e27892db2a266a81c85'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:23.430960 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a6a1791b-0ddb-4fa3-ba8e-7a3fcac77f3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b46c0aff591b4e27892db2a266a81c85'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:23.443442 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a6a1791b-0ddb-4fa3-ba8e-7a3fcac77f3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b46c0aff591b4e27892db2a266a81c85'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:23.444111 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a6a1791b-0ddb-4fa3-ba8e-7a3fcac77f3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b46c0aff591b4e27892db2a266a81c85'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:23.450005 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1177/2374] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:42:23 2022] DELETE /identity/v3/users/b46c0aff591b4e27892db2a266a81c85 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:23.454535 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac2e0b-a151-4dbc-856c-82273e8da776 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:23.454824 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac2e0b-a151-4dbc-856c-82273e8da776 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:23.455100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac2e0b-a151-4dbc-856c-82273e8da776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:23.475559 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-17ac2e0b-a151-4dbc-856c-82273e8da776 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:23.477236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-17ac2e0b-a151-4dbc-856c-82273e8da776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:23.553288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1198/2375] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:23.559587 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1791b-0ddb-4fa3-ba8e-7a3fcac77f3c admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:23.561194 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-a6a1791b-0ddb-4fa3-ba8e-7a3fcac77f3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:23.621175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1791b-0ddb-4fa3-ba8e-7a3fcac77f3c admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:23.624814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaaa5cb-3ad5-4421-8d7f-71b79a29a1c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:23.625495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaaa5cb-3ad5-4421-8d7f-71b79a29a1c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:23.625949 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaaa5cb-3ad5-4421-8d7f-71b79a29a1c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:23.626230 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaaa5cb-3ad5-4421-8d7f-71b79a29a1c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:23.635361 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaaa5cb-3ad5-4421-8d7f-71b79a29a1c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:23.636833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaaa5cb-3ad5-4421-8d7f-71b79a29a1c2 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:23.646516 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1178/2376] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:42:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:42:23.943968 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac2e0b-a151-4dbc-856c-82273e8da776 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:23.945488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-17ac2e0b-a151-4dbc-856c-82273e8da776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:23.959049 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac2e0b-a151-4dbc-856c-82273e8da776 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:23.963914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0fa65b-2a45-4438-a3a8-e488c93c8692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:23.964611 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fa65b-2a45-4438-a3a8-e488c93c8692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:23.964920 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fa65b-2a45-4438-a3a8-e488c93c8692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:23.965214 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fa65b-2a45-4438-a3a8-e488c93c8692 admin admin] PATH_INFO: `/v3/projects/06e8dbf44d8240a283ed46b979158f59` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:23.966392 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0fa65b-2a45-4438-a3a8-e488c93c8692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06e8dbf44d8240a283ed46b979158f59)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:23.969014 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0fa65b-2a45-4438-a3a8-e488c93c8692 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:24.016925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1f0fa65b-2a45-4438-a3a8-e488c93c8692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06e8dbf44d8240a283ed46b979158f59', 'request_id': 'req-1f0fa65b-2a45-4438-a3a8-e488c93c8692'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:24.024619 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1f0fa65b-2a45-4438-a3a8-e488c93c8692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06e8dbf44d8240a283ed46b979158f59', 'request_id': 'req-1f0fa65b-2a45-4438-a3a8-e488c93c8692'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:24.031048 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1f0fa65b-2a45-4438-a3a8-e488c93c8692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06e8dbf44d8240a283ed46b979158f59', 'request_id': 'req-1f0fa65b-2a45-4438-a3a8-e488c93c8692'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:24.042106 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1f0fa65b-2a45-4438-a3a8-e488c93c8692 admin admin] The token cache is being invalidate because project 06e8dbf44d8240a283ed46b979158f59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:42:24.042340 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1f0fa65b-2a45-4438-a3a8-e488c93c8692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:24.042972 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-1f0fa65b-2a45-4438-a3a8-e488c93c8692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:24.044491 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1199/2377] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:42:23 2022] DELETE /identity/v3/projects/06e8dbf44d8240a283ed46b979158f59 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:24.048494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaaa5cb-3ad5-4421-8d7f-71b79a29a1c2 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:24.058332 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaaa5cb-3ad5-4421-8d7f-71b79a29a1c2 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:24.117673 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031230a9-d261-440a-b657-c200bc7f8432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:24.118320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031230a9-d261-440a-b657-c200bc7f8432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:24.118605 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031230a9-d261-440a-b657-c200bc7f8432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:24.118886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031230a9-d261-440a-b657-c200bc7f8432 admin admin] PATH_INFO: `/v3/users/f6c22d0bb80d4d319b82358ec7a8b8ad` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:24.120090 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031230a9-d261-440a-b657-c200bc7f8432 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6c22d0bb80d4d319b82358ec7a8b8ad)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:24.122075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031230a9-d261-440a-b657-c200bc7f8432 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:24.168239 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-031230a9-d261-440a-b657-c200bc7f8432 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6c22d0bb80d4d319b82358ec7a8b8ad'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:24.175577 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-031230a9-d261-440a-b657-c200bc7f8432 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6c22d0bb80d4d319b82358ec7a8b8ad'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:24.182562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-031230a9-d261-440a-b657-c200bc7f8432 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6c22d0bb80d4d319b82358ec7a8b8ad'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:24.187729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:24.196078 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-031230a9-d261-440a-b657-c200bc7f8432 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6c22d0bb80d4d319b82358ec7a8b8ad'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:24.197071 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-031230a9-d261-440a-b657-c200bc7f8432 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6c22d0bb80d4d319b82358ec7a8b8ad'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:24.203517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1179/2378] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:42:24 2022] DELETE /identity/v3/users/f6c22d0bb80d4d319b82358ec7a8b8ad => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:24.533700 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0fa65b-2a45-4438-a3a8-e488c93c8692 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:24.535334 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-1f0fa65b-2a45-4438-a3a8-e488c93c8692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:24.549133 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0fa65b-2a45-4438-a3a8-e488c93c8692 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:24.562604 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:24.615758 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446bcac-1307-4229-9437-c5b95d8bcdf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:24.616453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446bcac-1307-4229-9437-c5b95d8bcdf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:24.616739 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446bcac-1307-4229-9437-c5b95d8bcdf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:24.617082 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446bcac-1307-4229-9437-c5b95d8bcdf1 admin admin] PATH_INFO: `/v3/projects/abdb1ffa71ac4a51a266f38698b6b366` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:24.618284 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446bcac-1307-4229-9437-c5b95d8bcdf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abdb1ffa71ac4a51a266f38698b6b366)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:24.620413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446bcac-1307-4229-9437-c5b95d8bcdf1 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:24.668403 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-9446bcac-1307-4229-9437-c5b95d8bcdf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abdb1ffa71ac4a51a266f38698b6b366', 'request_id': 'req-9446bcac-1307-4229-9437-c5b95d8bcdf1'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:24.676753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-9446bcac-1307-4229-9437-c5b95d8bcdf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abdb1ffa71ac4a51a266f38698b6b366', 'request_id': 'req-9446bcac-1307-4229-9437-c5b95d8bcdf1'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:24.683423 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-9446bcac-1307-4229-9437-c5b95d8bcdf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abdb1ffa71ac4a51a266f38698b6b366', 'request_id': 'req-9446bcac-1307-4229-9437-c5b95d8bcdf1'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:24.696053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-9446bcac-1307-4229-9437-c5b95d8bcdf1 admin admin] The token cache is being invalidate because project abdb1ffa71ac4a51a266f38698b6b366 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:42:24.696279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-9446bcac-1307-4229-9437-c5b95d8bcdf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:24.696956 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-9446bcac-1307-4229-9437-c5b95d8bcdf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:24.698472 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1200/2379] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:42:24 2022] DELETE /identity/v3/projects/abdb1ffa71ac4a51a266f38698b6b366 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:24.704131 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806db904-3075-44cb-91b9-1e839afbe6fa None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:24.704424 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806db904-3075-44cb-91b9-1e839afbe6fa None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:24.704706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806db904-3075-44cb-91b9-1e839afbe6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:24.725116 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-806db904-3075-44cb-91b9-1e839afbe6fa None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:24.726326 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-806db904-3075-44cb-91b9-1e839afbe6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:24.793226 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1180/2380] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:24.797344 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c72ca4-58d1-4439-9388-eb265c679049 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:24.797716 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c72ca4-58d1-4439-9388-eb265c679049 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:24.798002 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c72ca4-58d1-4439-9388-eb265c679049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:24.818936 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-60c72ca4-58d1-4439-9388-eb265c679049 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:24.820180 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-60c72ca4-58d1-4439-9388-eb265c679049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:24.828383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1201/2381] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:24.833262 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806db904-3075-44cb-91b9-1e839afbe6fa None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:24.843358 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806db904-3075-44cb-91b9-1e839afbe6fa None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:24.848924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09a9ee-5c18-420a-a14d-bc3d83434f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:24.849760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09a9ee-5c18-420a-a14d-bc3d83434f26 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:24.850146 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09a9ee-5c18-420a-a14d-bc3d83434f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:24.850312 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09a9ee-5c18-420a-a14d-bc3d83434f26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:24.851021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09a9ee-5c18-420a-a14d-bc3d83434f26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:42:24.851331 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09a9ee-5c18-420a-a14d-bc3d83434f26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:24.852961 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09a9ee-5c18-420a-a14d-bc3d83434f26 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:24.866168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1181/2382] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:42:24.877301 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c72ca4-58d1-4439-9388-eb265c679049 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:24.881331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c72ca4-58d1-4439-9388-eb265c679049 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:24.886188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d9dd3-892d-4769-a77c-e32dd296eb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:24.887168 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d9dd3-892d-4769-a77c-e32dd296eb64 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:24.887609 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d9dd3-892d-4769-a77c-e32dd296eb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:24.888040 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d9dd3-892d-4769-a77c-e32dd296eb64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:24.889096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d9dd3-892d-4769-a77c-e32dd296eb64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:24.891391 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d9dd3-892d-4769-a77c-e32dd296eb64 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:24.922961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1202/2383] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:42:24 2022] POST /identity/v3/projects => generated 391 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:42:24.927128 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09a9ee-5c18-420a-a14d-bc3d83434f26 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:24.941123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09a9ee-5c18-420a-a14d-bc3d83434f26 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:25.005469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246e5e5-fb62-43fe-bd90-cdb4858acbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:25.006401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246e5e5-fb62-43fe-bd90-cdb4858acbb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:25.006758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246e5e5-fb62-43fe-bd90-cdb4858acbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:25.007171 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246e5e5-fb62-43fe-bd90-cdb4858acbb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:25.007769 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6246e5e5-fb62-43fe-bd90-cdb4858acbb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:25.009754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6246e5e5-fb62-43fe-bd90-cdb4858acbb5 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:25.032932 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1182/2384] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:42:24 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:42:25.037166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d9dd3-892d-4769-a77c-e32dd296eb64 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:25.052288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d9dd3-892d-4769-a77c-e32dd296eb64 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:25.057805 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625a483-98dc-45ce-8ce1-bae45fead326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:25.058436 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625a483-98dc-45ce-8ce1-bae45fead326 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:25.058723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625a483-98dc-45ce-8ce1-bae45fead326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:25.059003 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625a483-98dc-45ce-8ce1-bae45fead326 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:25.059708 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625a483-98dc-45ce-8ce1-bae45fead326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:25.062306 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625a483-98dc-45ce-8ce1-bae45fead326 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:25.076377 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1203/2385] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:42:25.080786 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246e5e5-fb62-43fe-bd90-cdb4858acbb5 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:25.093789 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246e5e5-fb62-43fe-bd90-cdb4858acbb5 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:25.099107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867806d-ea27-4153-a523-84461c46fc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:25.099753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867806d-ea27-4153-a523-84461c46fc49 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:25.100038 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867806d-ea27-4153-a523-84461c46fc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:25.100325 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867806d-ea27-4153-a523-84461c46fc49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:25.101164 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867806d-ea27-4153-a523-84461c46fc49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:25.103039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867806d-ea27-4153-a523-84461c46fc49 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:25.123054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1183/2386] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:42:25.124485 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625a483-98dc-45ce-8ce1-bae45fead326 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:25.135125 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625a483-98dc-45ce-8ce1-bae45fead326 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:25.140940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962c9dd-7cb6-4292-95ad-1b94730ff9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:25.141947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962c9dd-7cb6-4292-95ad-1b94730ff9f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:25.142384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962c9dd-7cb6-4292-95ad-1b94730ff9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:25.142770 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962c9dd-7cb6-4292-95ad-1b94730ff9f1 admin admin] PATH_INFO: `/v3/projects/1aaed9a040f541d6afc9f990fe84b255/users/54d196cdb56844db9fb69cc2fbd1124d/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:25.162059 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962c9dd-7cb6-4292-95ad-1b94730ff9f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aaed9a040f541d6afc9f990fe84b255, user_id=54d196cdb56844db9fb69cc2fbd1124d, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:25.169033 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962c9dd-7cb6-4292-95ad-1b94730ff9f1 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:25.183529 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1204/2387] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:42:25 2022] PUT /identity/v3/projects/1aaed9a040f541d6afc9f990fe84b255/users/54d196cdb56844db9fb69cc2fbd1124d/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:42:25.195151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead301b0-8d2a-4d14-b466-e88a511b2d52 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:25.195445 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead301b0-8d2a-4d14-b466-e88a511b2d52 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:25.195726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead301b0-8d2a-4d14-b466-e88a511b2d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:25.231529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-ead301b0-8d2a-4d14-b466-e88a511b2d52 None None] MFA Rules not processed for user `54d196cdb56844db9fb69cc2fbd1124d`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:25.233605 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ead301b0-8d2a-4d14-b466-e88a511b2d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:25.290067 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:25.319490 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1184/2388] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:25.324206 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962c9dd-7cb6-4292-95ad-1b94730ff9f1 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:25.372373 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:25.395044 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962c9dd-7cb6-4292-95ad-1b94730ff9f1 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:25.400237 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba86db-e6dc-4f7a-a755-47a330400ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:25.401001 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba86db-e6dc-4f7a-a755-47a330400ffd admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:25.401390 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba86db-e6dc-4f7a-a755-47a330400ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:25.401755 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba86db-e6dc-4f7a-a755-47a330400ffd admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:25.402510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ba86db-e6dc-4f7a-a755-47a330400ffd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:25.405442 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ba86db-e6dc-4f7a-a755-47a330400ffd admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:25.439365 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1205/2389] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:42:25 2022] POST /identity/v3/projects => generated 389 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:42:25.443549 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead301b0-8d2a-4d14-b466-e88a511b2d52 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:25.458814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0c6e9-5d1e-40e0-94ac-9f54e585638d None None] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:25.459131 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0c6e9-5d1e-40e0-94ac-9f54e585638d None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:25.459626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0c6e9-5d1e-40e0-94ac-9f54e585638d None None] PATH_INFO: `/` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:25.461488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1206/2390] 10.222.0.74 () {54 vars in 921 bytes} [Wed Jul 27 16:42:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:42:25.461719 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead301b0-8d2a-4d14-b466-e88a511b2d52 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:25.469751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8610513-30fa-4d7a-8210-0b5789934451 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:25.469982 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8610513-30fa-4d7a-8210-0b5789934451 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:25.470378 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8610513-30fa-4d7a-8210-0b5789934451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:25.498548 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-b8610513-30fa-4d7a-8210-0b5789934451 None None] MFA Rules not processed for user `56dfafd15e284b7986ba07dd3819bf65`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:25.499976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b8610513-30fa-4d7a-8210-0b5789934451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:25.531497 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269dfb29-b790-4d33-8c32-a2913f85c224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:25.532482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269dfb29-b790-4d33-8c32-a2913f85c224 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:25.532482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269dfb29-b790-4d33-8c32-a2913f85c224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:25.532657 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269dfb29-b790-4d33-8c32-a2913f85c224 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:25.533358 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269dfb29-b790-4d33-8c32-a2913f85c224 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:25.535321 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269dfb29-b790-4d33-8c32-a2913f85c224 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:25.558833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1185/2391] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:42:25 2022] POST /identity/v3/users => generated 301 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:42:25.563251 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269dfb29-b790-4d33-8c32-a2913f85c224 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:25.575666 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269dfb29-b790-4d33-8c32-a2913f85c224 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:25.581758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41246c72-86dd-4797-a2f5-af0f34d9e4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:25.582445 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41246c72-86dd-4797-a2f5-af0f34d9e4a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:25.582798 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41246c72-86dd-4797-a2f5-af0f34d9e4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:25.583198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41246c72-86dd-4797-a2f5-af0f34d9e4a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:25.584199 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41246c72-86dd-4797-a2f5-af0f34d9e4a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:25.585976 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41246c72-86dd-4797-a2f5-af0f34d9e4a8 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:25.598027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1207/2392] 10.222.0.74 () {58 vars in 1018 bytes} [Wed Jul 27 16:42:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:42:25.599609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1186/2393] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:42:25.605620 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8610513-30fa-4d7a-8210-0b5789934451 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:25.611968 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41246c72-86dd-4797-a2f5-af0f34d9e4a8 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:25.614023 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-41246c72-86dd-4797-a2f5-af0f34d9e4a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:25.631374 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8610513-30fa-4d7a-8210-0b5789934451 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:25.636346 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220561d-1015-48ba-b398-e335020aea5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:25.637086 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220561d-1015-48ba-b398-e335020aea5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:25.637369 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220561d-1015-48ba-b398-e335020aea5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:25.637640 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220561d-1015-48ba-b398-e335020aea5c admin admin] PATH_INFO: `/v3/projects/4fefc36a97264b3bad66bd0ed6673ec9/users/03d9acaaf0784b048ec669d8bef4a6d5/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:25.654435 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220561d-1015-48ba-b398-e335020aea5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fefc36a97264b3bad66bd0ed6673ec9, user_id=03d9acaaf0784b048ec669d8bef4a6d5, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:25.657434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220561d-1015-48ba-b398-e335020aea5c admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:25.671372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1208/2394] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:42:25 2022] PUT /identity/v3/projects/4fefc36a97264b3bad66bd0ed6673ec9/users/03d9acaaf0784b048ec669d8bef4a6d5/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:25.675124 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220561d-1015-48ba-b398-e335020aea5c admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:25.677600 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41246c72-86dd-4797-a2f5-af0f34d9e4a8 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:25.727467 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f2f9a-f671-48a2-ab74-45399cd73ff4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c50c2d1781b04a5fa8be81bce9205747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:25.728014 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f2f9a-f671-48a2-ab74-45399cd73ff4 service glance] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:25.728293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f2f9a-f671-48a2-ab74-45399cd73ff4 service glance] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:25.728603 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f2f9a-f671-48a2-ab74-45399cd73ff4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:25.736818 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220561d-1015-48ba-b398-e335020aea5c admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:25.741463 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277172b2-6b1b-4392-9a44-19fafe90bf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:25.742065 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277172b2-6b1b-4392-9a44-19fafe90bf3d admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:25.742341 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277172b2-6b1b-4392-9a44-19fafe90bf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:25.742616 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277172b2-6b1b-4392-9a44-19fafe90bf3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:25.743303 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277172b2-6b1b-4392-9a44-19fafe90bf3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:25.744929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277172b2-6b1b-4392-9a44-19fafe90bf3d admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:25.755135 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1209/2395] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:42:25.759097 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277172b2-6b1b-4392-9a44-19fafe90bf3d admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:25.761157 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:25.769339 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277172b2-6b1b-4392-9a44-19fafe90bf3d admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:25.773905 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53537a18-db46-4f2f-9382-b2afd06b038b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:25.774556 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53537a18-db46-4f2f-9382-b2afd06b038b admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:25.775156 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53537a18-db46-4f2f-9382-b2afd06b038b admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:25.775156 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53537a18-db46-4f2f-9382-b2afd06b038b admin admin] PATH_INFO: `/v3/projects/4fefc36a97264b3bad66bd0ed6673ec9/users/03d9acaaf0784b048ec669d8bef4a6d5/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:25.777114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53537a18-db46-4f2f-9382-b2afd06b038b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fefc36a97264b3bad66bd0ed6673ec9, user_id=03d9acaaf0784b048ec669d8bef4a6d5, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:25.779812 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53537a18-db46-4f2f-9382-b2afd06b038b admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:25.789937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1210/2396] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:42:25 2022] PUT /identity/v3/projects/4fefc36a97264b3bad66bd0ed6673ec9/users/03d9acaaf0784b048ec669d8bef4a6d5/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:42:25.800770 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bdb51-2c7b-4a72-9022-a5e64459cf77 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:25.800947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f2f9a-f671-48a2-ab74-45399cd73ff4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:25.801090 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bdb51-2c7b-4a72-9022-a5e64459cf77 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:25.801351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bdb51-2c7b-4a72-9022-a5e64459cf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:25.802453 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f2f9a-f671-48a2-ab74-45399cd73ff4 service glance] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:25.815410 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1187/2397] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:42:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:42:25.829914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-b01bdb51-2c7b-4a72-9022-a5e64459cf77 None None] MFA Rules not processed for user `03d9acaaf0784b048ec669d8bef4a6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:25.831288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b01bdb51-2c7b-4a72-9022-a5e64459cf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:25.838800 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:25.885546 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd215b5-058f-41db-adf3-227bc8462ab2 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:25.885938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd215b5-058f-41db-adf3-227bc8462ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:25.886283 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd215b5-058f-41db-adf3-227bc8462ab2 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:25.887733 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1188/2398] 10.222.0.74 () {54 vars in 921 bytes} [Wed Jul 27 16:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 16:42:25.892327 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e033a-6b6d-4f99-af18-40c550db1db9 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:25.892701 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e033a-6b6d-4f99-af18-40c550db1db9 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:25.893104 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e033a-6b6d-4f99-af18-40c550db1db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:25.906675 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1211/2399] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:42:25.914019 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-751e033a-6b6d-4f99-af18-40c550db1db9 None None] MFA Rules not processed for user `193dd9fee1844275ad0dab602e93df11`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:25.915242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-751e033a-6b6d-4f99-af18-40c550db1db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:25.918352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01bdb51-2c7b-4a72-9022-a5e64459cf77 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:25.919829 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b01bdb51-2c7b-4a72-9022-a5e64459cf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:25.966376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1189/2400] 10.222.0.74 () {58 vars in 1018 bytes} [Wed Jul 27 16:42:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:42:25.977376 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01bdb51-2c7b-4a72-9022-a5e64459cf77 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:25.979871 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e033a-6b6d-4f99-af18-40c550db1db9 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:25.981205 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-751e033a-6b6d-4f99-af18-40c550db1db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:25.981339 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27808fc-7d38-479c-b539-576ce66d2358 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:25.981912 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27808fc-7d38-479c-b539-576ce66d2358 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:25.982191 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27808fc-7d38-479c-b539-576ce66d2358 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:25.982463 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27808fc-7d38-479c-b539-576ce66d2358 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:25.992208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e033a-6b6d-4f99-af18-40c550db1db9 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:25.992416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27808fc-7d38-479c-b539-576ce66d2358 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:25.993890 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27808fc-7d38-479c-b539-576ce66d2358 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:25.996069 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059813e5-874d-4a89-8067-c8c8c2461f04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:25.996674 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059813e5-874d-4a89-8067-c8c8c2461f04 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:25.996982 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059813e5-874d-4a89-8067-c8c8c2461f04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:25.997265 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059813e5-874d-4a89-8067-c8c8c2461f04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:26.003611 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1212/2401] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:42:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:42:26.005290 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059813e5-874d-4a89-8067-c8c8c2461f04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:26.006817 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059813e5-874d-4a89-8067-c8c8c2461f04 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:26.015348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1190/2402] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:42:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:42:26.052292 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27808fc-7d38-479c-b539-576ce66d2358 service cinder] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:26.053735 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-a27808fc-7d38-479c-b539-576ce66d2358 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:26.064934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27808fc-7d38-479c-b539-576ce66d2358 service cinder] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:26.068574 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfbef8e-b568-4f88-acbc-bfca016bd73f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:26.069230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbef8e-b568-4f88-acbc-bfca016bd73f service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:26.069515 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbef8e-b568-4f88-acbc-bfca016bd73f service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:26.069809 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbef8e-b568-4f88-acbc-bfca016bd73f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:26.102858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f940a5-25b8-4905-be34-bc8b29f4f368 None None] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:26.103342 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f940a5-25b8-4905-be34-bc8b29f4f368 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:26.103425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f940a5-25b8-4905-be34-bc8b29f4f368 None None] PATH_INFO: `/` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:26.104742 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1191/2403] 10.222.0.74 () {54 vars in 921 bytes} [Wed Jul 27 16:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 16:42:26.111238 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfc2ba-5bdd-4665-8d37-1e41ea18d6ca None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:26.111610 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfc2ba-5bdd-4665-8d37-1e41ea18d6ca None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:26.111966 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecfc2ba-5bdd-4665-8d37-1e41ea18d6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:26.132070 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-5ecfc2ba-5bdd-4665-8d37-1e41ea18d6ca None None] MFA Rules not processed for user `56dfafd15e284b7986ba07dd3819bf65`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:26.133545 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-5ecfc2ba-5bdd-4665-8d37-1e41ea18d6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:26.134572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfbef8e-b568-4f88-acbc-bfca016bd73f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:26.136009 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfbef8e-b568-4f88-acbc-bfca016bd73f service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:26.145551 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1213/2404] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:42:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:42:26.207423 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1192/2405] 10.222.0.74 () {58 vars in 1018 bytes} [Wed Jul 27 16:42:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:42:26.219499 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfbef8e-b568-4f88-acbc-bfca016bd73f service cinder] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:26.221709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-2cfbef8e-b568-4f88-acbc-bfca016bd73f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:26.235488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfbef8e-b568-4f88-acbc-bfca016bd73f service cinder] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:26.239287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629ed71-6a76-4054-a257-5f04abe27f50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:26.239925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629ed71-6a76-4054-a257-5f04abe27f50 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:26.240207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629ed71-6a76-4054-a257-5f04abe27f50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:26.240477 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629ed71-6a76-4054-a257-5f04abe27f50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:26.251129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629ed71-6a76-4054-a257-5f04abe27f50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:26.252654 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629ed71-6a76-4054-a257-5f04abe27f50 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:26.277002 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1214/2406] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:42:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:42:27.980152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5b7c7-f812-4187-a4d3-c84c85206e15 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:27.980619 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5b7c7-f812-4187-a4d3-c84c85206e15 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:27.980711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5b7c7-f812-4187-a4d3-c84c85206e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:28.004769 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-b6b5b7c7-f812-4187-a4d3-c84c85206e15 None None] MFA Rules not processed for user `07d6bffc69674103b4c6540598a780b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:28.006700 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b6b5b7c7-f812-4187-a4d3-c84c85206e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:28.054373 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:28.068598 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1193/2407] 10.222.0.74 () {58 vars in 1009 bytes} [Wed Jul 27 16:42:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:28.093664 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629ed71-6a76-4054-a257-5f04abe27f50 service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:28.095311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-6629ed71-6a76-4054-a257-5f04abe27f50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:28.133210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:28.177710 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629ed71-6a76-4054-a257-5f04abe27f50 service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:28.183111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cfafd1-d202-4ce4-9835-24b4217f01b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb45ded9f75e4fa2973b564f83bdfaf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:28.183677 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfafd1-d202-4ce4-9835-24b4217f01b0 service swift] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:28.183987 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfafd1-d202-4ce4-9835-24b4217f01b0 service swift] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:28.184266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfafd1-d202-4ce4-9835-24b4217f01b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:28.193198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cfafd1-d202-4ce4-9835-24b4217f01b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:28.194793 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cfafd1-d202-4ce4-9835-24b4217f01b0 service swift] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:28.203941 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1215/2408] 10.222.0.74 () {60 vars in 1384 bytes} [Wed Jul 27 16:42:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 16:42:29.673054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5b7c7-f812-4187-a4d3-c84c85206e15 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:29.742373 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5b7c7-f812-4187-a4d3-c84c85206e15 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:29.747385 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92698bbc-24b6-4bbd-8d85-56be7031f043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:29.748188 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92698bbc-24b6-4bbd-8d85-56be7031f043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:29.748489 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92698bbc-24b6-4bbd-8d85-56be7031f043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:29.748765 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92698bbc-24b6-4bbd-8d85-56be7031f043 admin admin] PATH_INFO: `/v3/users/54d196cdb56844db9fb69cc2fbd1124d` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:29.750150 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92698bbc-24b6-4bbd-8d85-56be7031f043 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54d196cdb56844db9fb69cc2fbd1124d)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:29.752095 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92698bbc-24b6-4bbd-8d85-56be7031f043 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:29.809322 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-92698bbc-24b6-4bbd-8d85-56be7031f043 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54d196cdb56844db9fb69cc2fbd1124d'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:29.818316 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-92698bbc-24b6-4bbd-8d85-56be7031f043 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54d196cdb56844db9fb69cc2fbd1124d'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:29.826869 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-92698bbc-24b6-4bbd-8d85-56be7031f043 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54d196cdb56844db9fb69cc2fbd1124d'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:29.839856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-92698bbc-24b6-4bbd-8d85-56be7031f043 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54d196cdb56844db9fb69cc2fbd1124d'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:29.840687 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-92698bbc-24b6-4bbd-8d85-56be7031f043 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54d196cdb56844db9fb69cc2fbd1124d'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:29.848451 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1194/2409] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:42:29 2022] DELETE /identity/v3/users/54d196cdb56844db9fb69cc2fbd1124d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:29.853038 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57982f-b083-449e-aba2-1560865d4c63 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:29.853518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57982f-b083-449e-aba2-1560865d4c63 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:29.853627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57982f-b083-449e-aba2-1560865d4c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:29.873949 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-2e57982f-b083-449e-aba2-1560865d4c63 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:29.875213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-2e57982f-b083-449e-aba2-1560865d4c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:29.937439 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1216/2410] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:29.946987 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92698bbc-24b6-4bbd-8d85-56be7031f043 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:29.948411 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-92698bbc-24b6-4bbd-8d85-56be7031f043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:30.010586 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92698bbc-24b6-4bbd-8d85-56be7031f043 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:30.014879 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8744dc-f591-49c2-9c35-a280497fc4e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:30.015564 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8744dc-f591-49c2-9c35-a280497fc4e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:30.015915 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8744dc-f591-49c2-9c35-a280497fc4e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:30.016290 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8744dc-f591-49c2-9c35-a280497fc4e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:30.026202 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8744dc-f591-49c2-9c35-a280497fc4e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:30.027639 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8744dc-f591-49c2-9c35-a280497fc4e4 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:30.038625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1195/2411] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:42:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:42:30.331267 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57982f-b083-449e-aba2-1560865d4c63 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:30.332976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-2e57982f-b083-449e-aba2-1560865d4c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:30.350595 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57982f-b083-449e-aba2-1560865d4c63 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:30.361616 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a02cac-97b4-475e-8e3e-4182832d22ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:30.362531 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a02cac-97b4-475e-8e3e-4182832d22ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:30.365320 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a02cac-97b4-475e-8e3e-4182832d22ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:30.365979 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a02cac-97b4-475e-8e3e-4182832d22ef admin admin] PATH_INFO: `/v3/projects/1aaed9a040f541d6afc9f990fe84b255` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:30.368050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a02cac-97b4-475e-8e3e-4182832d22ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aaed9a040f541d6afc9f990fe84b255)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:30.376558 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a02cac-97b4-475e-8e3e-4182832d22ef admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:30.453195 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-45a02cac-97b4-475e-8e3e-4182832d22ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aaed9a040f541d6afc9f990fe84b255', 'request_id': 'req-45a02cac-97b4-475e-8e3e-4182832d22ef'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:30.463097 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-45a02cac-97b4-475e-8e3e-4182832d22ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aaed9a040f541d6afc9f990fe84b255', 'request_id': 'req-45a02cac-97b4-475e-8e3e-4182832d22ef'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:30.469846 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-45a02cac-97b4-475e-8e3e-4182832d22ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aaed9a040f541d6afc9f990fe84b255', 'request_id': 'req-45a02cac-97b4-475e-8e3e-4182832d22ef'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:30.483025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-45a02cac-97b4-475e-8e3e-4182832d22ef admin admin] The token cache is being invalidate because project 1aaed9a040f541d6afc9f990fe84b255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:42:30.483322 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-45a02cac-97b4-475e-8e3e-4182832d22ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:30.483983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-45a02cac-97b4-475e-8e3e-4182832d22ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:30.485585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1217/2412] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:42:30 2022] DELETE /identity/v3/projects/1aaed9a040f541d6afc9f990fe84b255 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:30.489548 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8744dc-f591-49c2-9c35-a280497fc4e4 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:30.503190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8744dc-f591-49c2-9c35-a280497fc4e4 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:30.515765 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:30.591849 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ab21f-e0a2-4797-bc1a-f3dd6a6fc7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:30.593448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ab21f-e0a2-4797-bc1a-f3dd6a6fc7cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:30.593448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ab21f-e0a2-4797-bc1a-f3dd6a6fc7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:30.594593 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ab21f-e0a2-4797-bc1a-f3dd6a6fc7cf admin admin] PATH_INFO: `/v3/users/03d9acaaf0784b048ec669d8bef4a6d5` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:30.596748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ab21f-e0a2-4797-bc1a-f3dd6a6fc7cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=03d9acaaf0784b048ec669d8bef4a6d5)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:30.599653 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ab21f-e0a2-4797-bc1a-f3dd6a6fc7cf admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:30.694737 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-de7ab21f-e0a2-4797-bc1a-f3dd6a6fc7cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03d9acaaf0784b048ec669d8bef4a6d5'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:30.702974 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-de7ab21f-e0a2-4797-bc1a-f3dd6a6fc7cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03d9acaaf0784b048ec669d8bef4a6d5'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:30.711296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-de7ab21f-e0a2-4797-bc1a-f3dd6a6fc7cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03d9acaaf0784b048ec669d8bef4a6d5'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:30.725784 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-de7ab21f-e0a2-4797-bc1a-f3dd6a6fc7cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03d9acaaf0784b048ec669d8bef4a6d5'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:30.726517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-de7ab21f-e0a2-4797-bc1a-f3dd6a6fc7cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03d9acaaf0784b048ec669d8bef4a6d5'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:30.733471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1196/2413] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:42:30 2022] DELETE /identity/v3/users/03d9acaaf0784b048ec669d8bef4a6d5 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:31.083922 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a02cac-97b4-475e-8e3e-4182832d22ef admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:31.085387 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-45a02cac-97b4-475e-8e3e-4182832d22ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:31.111719 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a02cac-97b4-475e-8e3e-4182832d22ef admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:31.168141 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:31.183286 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd85e72-f3cd-4598-91ef-2dc6b73b40fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:31.184386 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd85e72-f3cd-4598-91ef-2dc6b73b40fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:31.185049 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd85e72-f3cd-4598-91ef-2dc6b73b40fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:31.185793 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd85e72-f3cd-4598-91ef-2dc6b73b40fb admin admin] PATH_INFO: `/v3/projects/4fefc36a97264b3bad66bd0ed6673ec9` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:31.187417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd85e72-f3cd-4598-91ef-2dc6b73b40fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fefc36a97264b3bad66bd0ed6673ec9)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:31.189579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd85e72-f3cd-4598-91ef-2dc6b73b40fb admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:31.239753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-2dd85e72-f3cd-4598-91ef-2dc6b73b40fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fefc36a97264b3bad66bd0ed6673ec9', 'request_id': 'req-2dd85e72-f3cd-4598-91ef-2dc6b73b40fb'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:31.248708 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-2dd85e72-f3cd-4598-91ef-2dc6b73b40fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fefc36a97264b3bad66bd0ed6673ec9', 'request_id': 'req-2dd85e72-f3cd-4598-91ef-2dc6b73b40fb'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:31.255336 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-2dd85e72-f3cd-4598-91ef-2dc6b73b40fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fefc36a97264b3bad66bd0ed6673ec9', 'request_id': 'req-2dd85e72-f3cd-4598-91ef-2dc6b73b40fb'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:31.267963 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-2dd85e72-f3cd-4598-91ef-2dc6b73b40fb admin admin] The token cache is being invalidate because project 4fefc36a97264b3bad66bd0ed6673ec9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:42:31.268387 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-2dd85e72-f3cd-4598-91ef-2dc6b73b40fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:31.269185 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-2dd85e72-f3cd-4598-91ef-2dc6b73b40fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:31.271315 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1218/2414] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:42:31 2022] DELETE /identity/v3/projects/4fefc36a97264b3bad66bd0ed6673ec9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:31.276386 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7925d-f993-42d9-b603-d7e46f6ca778 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:31.276875 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7925d-f993-42d9-b603-d7e46f6ca778 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:31.277375 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7925d-f993-42d9-b603-d7e46f6ca778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:31.298602 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-51b7925d-f993-42d9-b603-d7e46f6ca778 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:31.299838 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-51b7925d-f993-42d9-b603-d7e46f6ca778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:31.363009 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1197/2415] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:42:31.371235 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258a785-5055-448d-99ad-42cebe80c7ff None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:31.371477 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258a785-5055-448d-99ad-42cebe80c7ff None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:31.371758 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258a785-5055-448d-99ad-42cebe80c7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:31.392718 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-1258a785-5055-448d-99ad-42cebe80c7ff None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:31.394036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-1258a785-5055-448d-99ad-42cebe80c7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:31.402036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1219/2416] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:31.406867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7925d-f993-42d9-b603-d7e46f6ca778 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:31.416655 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7925d-f993-42d9-b603-d7e46f6ca778 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:31.421322 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c8f72-fb0a-4cc9-b79d-1703184f89a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:31.421931 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c8f72-fb0a-4cc9-b79d-1703184f89a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:31.422215 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c8f72-fb0a-4cc9-b79d-1703184f89a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:31.422492 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c8f72-fb0a-4cc9-b79d-1703184f89a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:31.423182 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c8f72-fb0a-4cc9-b79d-1703184f89a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:42:31.423489 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c8f72-fb0a-4cc9-b79d-1703184f89a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:31.425138 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c8f72-fb0a-4cc9-b79d-1703184f89a4 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:31.437172 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1198/2417] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:42:31.441999 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258a785-5055-448d-99ad-42cebe80c7ff None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:31.452311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258a785-5055-448d-99ad-42cebe80c7ff None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:31.457319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb0d5c-6f08-4419-b931-fcfc5beacef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:31.458176 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb0d5c-6f08-4419-b931-fcfc5beacef0 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:31.458537 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb0d5c-6f08-4419-b931-fcfc5beacef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:31.458879 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb0d5c-6f08-4419-b931-fcfc5beacef0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:31.459637 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb0d5c-6f08-4419-b931-fcfc5beacef0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:31.461681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb0d5c-6f08-4419-b931-fcfc5beacef0 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:31.492236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1220/2418] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:42:31 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:42:31.496408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c8f72-fb0a-4cc9-b79d-1703184f89a4 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:31.508306 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c8f72-fb0a-4cc9-b79d-1703184f89a4 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:31.554165 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:31.607706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00177fd5-8ea0-495e-a7ef-c36423a397e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:31.608346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00177fd5-8ea0-495e-a7ef-c36423a397e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:31.608631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00177fd5-8ea0-495e-a7ef-c36423a397e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:31.608916 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00177fd5-8ea0-495e-a7ef-c36423a397e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:31.609636 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00177fd5-8ea0-495e-a7ef-c36423a397e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:31.611614 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00177fd5-8ea0-495e-a7ef-c36423a397e0 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:31.647348 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1199/2419] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:42:31 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:42:31.650960 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb0d5c-6f08-4419-b931-fcfc5beacef0 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:31.667206 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb0d5c-6f08-4419-b931-fcfc5beacef0 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:31.687011 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86e8dc-a700-464d-99a6-15144683d395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:31.687658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86e8dc-a700-464d-99a6-15144683d395 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:31.687941 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86e8dc-a700-464d-99a6-15144683d395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:31.688216 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86e8dc-a700-464d-99a6-15144683d395 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:31.688954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a86e8dc-a700-464d-99a6-15144683d395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:31.690829 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a86e8dc-a700-464d-99a6-15144683d395 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:31.703251 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1221/2420] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:42:31.715308 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00177fd5-8ea0-495e-a7ef-c36423a397e0 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:31.726933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00177fd5-8ea0-495e-a7ef-c36423a397e0 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:31.731711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3b526-dccf-4a83-8940-94ac51e5a275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:31.732346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3b526-dccf-4a83-8940-94ac51e5a275 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:31.732638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3b526-dccf-4a83-8940-94ac51e5a275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:31.732935 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3b526-dccf-4a83-8940-94ac51e5a275 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:31.733638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e3b526-dccf-4a83-8940-94ac51e5a275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:31.735251 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e3b526-dccf-4a83-8940-94ac51e5a275 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:31.750032 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1200/2421] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:42:31.752010 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86e8dc-a700-464d-99a6-15144683d395 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:31.765544 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86e8dc-a700-464d-99a6-15144683d395 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:31.771574 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d208864-88f8-461a-9711-74121e0c569b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:31.772478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d208864-88f8-461a-9711-74121e0c569b admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:31.772768 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d208864-88f8-461a-9711-74121e0c569b admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:31.773120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d208864-88f8-461a-9711-74121e0c569b admin admin] PATH_INFO: `/v3/projects/09797e92acc04ffdaf58f6ac35fa105f/users/d705117cf1d943c09302431aab73ab8a/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:31.789870 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d208864-88f8-461a-9711-74121e0c569b admin admin] RBAC: Authorizing `identity:create_grant(project_id=09797e92acc04ffdaf58f6ac35fa105f, user_id=d705117cf1d943c09302431aab73ab8a, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:31.792727 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d208864-88f8-461a-9711-74121e0c569b admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:31.806642 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1222/2422] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:42:31 2022] PUT /identity/v3/projects/09797e92acc04ffdaf58f6ac35fa105f/users/d705117cf1d943c09302431aab73ab8a/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:42:31.818248 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c3900-2594-4a7e-8294-2d6bd30541d9 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:31.818637 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c3900-2594-4a7e-8294-2d6bd30541d9 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:31.818849 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c3900-2594-4a7e-8294-2d6bd30541d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:31.852000 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-fd4c3900-2594-4a7e-8294-2d6bd30541d9 None None] MFA Rules not processed for user `d705117cf1d943c09302431aab73ab8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:31.853288 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-fd4c3900-2594-4a7e-8294-2d6bd30541d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:31.921156 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1201/2423] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:31.933178 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d208864-88f8-461a-9711-74121e0c569b admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:31.935021 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-8d208864-88f8-461a-9711-74121e0c569b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:32.006638 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d208864-88f8-461a-9711-74121e0c569b admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:32.010305 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec6552-d980-49c2-8c69-f703f51ef63c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c50c2d1781b04a5fa8be81bce9205747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:32.010916 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec6552-d980-49c2-8c69-f703f51ef63c service glance] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:32.011195 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec6552-d980-49c2-8c69-f703f51ef63c service glance] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:32.011469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec6552-d980-49c2-8c69-f703f51ef63c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:32.020660 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec6552-d980-49c2-8c69-f703f51ef63c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:32.022118 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec6552-d980-49c2-8c69-f703f51ef63c service glance] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:32.031793 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1223/2424] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:42:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:42:41.917417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=57,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:42.026862 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:46.711169 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080b7d0-133e-4745-8200-b586bc68e490 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:46.711675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080b7d0-133e-4745-8200-b586bc68e490 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:46.711784 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080b7d0-133e-4745-8200-b586bc68e490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:46.732351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-9080b7d0-133e-4745-8200-b586bc68e490 None None] MFA Rules not processed for user `07d6bffc69674103b4c6540598a780b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:46.733770 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9080b7d0-133e-4745-8200-b586bc68e490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:46.807305 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1202/2425] 10.222.0.74 () {58 vars in 967 bytes} [Wed Jul 27 16:42:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:46.820584 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec6552-d980-49c2-8c69-f703f51ef63c service glance] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:46.822133 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-fbec6552-d980-49c2-8c69-f703f51ef63c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:46.900240 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec6552-d980-49c2-8c69-f703f51ef63c service glance] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:46.905221 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8661d9-73ed-4150-8427-37adce5acc16 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb45ded9f75e4fa2973b564f83bdfaf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:46.906976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8661d9-73ed-4150-8427-37adce5acc16 service swift] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:46.906976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8661d9-73ed-4150-8427-37adce5acc16 service swift] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:46.906976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8661d9-73ed-4150-8427-37adce5acc16 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:46.914794 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8661d9-73ed-4150-8427-37adce5acc16 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:46.916363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8661d9-73ed-4150-8427-37adce5acc16 service swift] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:46.924832 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1224/2426] 10.222.0.74 () {60 vars in 1384 bytes} [Wed Jul 27 16:42:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 16:42:50.966022 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080b7d0-133e-4745-8200-b586bc68e490 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:51.037195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080b7d0-133e-4745-8200-b586bc68e490 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:51.041842 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e8ae4-143a-4686-ab62-0108ae2594e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:51.042684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e8ae4-143a-4686-ab62-0108ae2594e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:51.043079 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e8ae4-143a-4686-ab62-0108ae2594e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:51.045145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e8ae4-143a-4686-ab62-0108ae2594e3 admin admin] PATH_INFO: `/v3/users/d705117cf1d943c09302431aab73ab8a` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:51.045546 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e8ae4-143a-4686-ab62-0108ae2594e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d705117cf1d943c09302431aab73ab8a)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:51.047369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e8ae4-143a-4686-ab62-0108ae2594e3 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:51.100217 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-203e8ae4-143a-4686-ab62-0108ae2594e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd705117cf1d943c09302431aab73ab8a'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:51.108135 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-203e8ae4-143a-4686-ab62-0108ae2594e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd705117cf1d943c09302431aab73ab8a'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:51.115230 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-203e8ae4-143a-4686-ab62-0108ae2594e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd705117cf1d943c09302431aab73ab8a'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:51.127304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-203e8ae4-143a-4686-ab62-0108ae2594e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd705117cf1d943c09302431aab73ab8a'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:51.127972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-203e8ae4-143a-4686-ab62-0108ae2594e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd705117cf1d943c09302431aab73ab8a'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:51.133753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1203/2427] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:42:50 2022] DELETE /identity/v3/users/d705117cf1d943c09302431aab73ab8a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:51.138219 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b861b2-3047-4da5-938a-c60df12309ab None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:51.138672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b861b2-3047-4da5-938a-c60df12309ab None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:51.138784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b861b2-3047-4da5-938a-c60df12309ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:51.158571 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-26b861b2-3047-4da5-938a-c60df12309ab None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:51.159796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-26b861b2-3047-4da5-938a-c60df12309ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:51.219548 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1225/2428] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:51.229619 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e8ae4-143a-4686-ab62-0108ae2594e3 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:51.231059 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-203e8ae4-143a-4686-ab62-0108ae2594e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:51.261389 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:51.295795 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e8ae4-143a-4686-ab62-0108ae2594e3 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:51.299585 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347c1e2-5aaf-4ad7-9e8f-e15dcea4ca48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:51.300200 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347c1e2-5aaf-4ad7-9e8f-e15dcea4ca48 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:51.300476 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347c1e2-5aaf-4ad7-9e8f-e15dcea4ca48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:51.300743 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347c1e2-5aaf-4ad7-9e8f-e15dcea4ca48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:51.310304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347c1e2-5aaf-4ad7-9e8f-e15dcea4ca48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:51.311753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347c1e2-5aaf-4ad7-9e8f-e15dcea4ca48 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:51.322852 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1204/2429] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:42:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:42:51.691268 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b861b2-3047-4da5-938a-c60df12309ab None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:51.693515 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-26b861b2-3047-4da5-938a-c60df12309ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:51.709046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b861b2-3047-4da5-938a-c60df12309ab None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:51.714212 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff4773-5cdb-4ddf-abe3-9e2ee14019a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:51.714866 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff4773-5cdb-4ddf-abe3-9e2ee14019a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:51.715148 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff4773-5cdb-4ddf-abe3-9e2ee14019a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:51.715506 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff4773-5cdb-4ddf-abe3-9e2ee14019a6 admin admin] PATH_INFO: `/v3/projects/09797e92acc04ffdaf58f6ac35fa105f` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:51.716622 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff4773-5cdb-4ddf-abe3-9e2ee14019a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09797e92acc04ffdaf58f6ac35fa105f)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:51.718736 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff4773-5cdb-4ddf-abe3-9e2ee14019a6 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:51.775907 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-59ff4773-5cdb-4ddf-abe3-9e2ee14019a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09797e92acc04ffdaf58f6ac35fa105f', 'request_id': 'req-59ff4773-5cdb-4ddf-abe3-9e2ee14019a6'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:51.784089 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-59ff4773-5cdb-4ddf-abe3-9e2ee14019a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09797e92acc04ffdaf58f6ac35fa105f', 'request_id': 'req-59ff4773-5cdb-4ddf-abe3-9e2ee14019a6'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:51.790250 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-59ff4773-5cdb-4ddf-abe3-9e2ee14019a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09797e92acc04ffdaf58f6ac35fa105f', 'request_id': 'req-59ff4773-5cdb-4ddf-abe3-9e2ee14019a6'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:51.802504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-59ff4773-5cdb-4ddf-abe3-9e2ee14019a6 admin admin] The token cache is being invalidate because project 09797e92acc04ffdaf58f6ac35fa105f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:42:51.802805 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-59ff4773-5cdb-4ddf-abe3-9e2ee14019a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:51.803467 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-59ff4773-5cdb-4ddf-abe3-9e2ee14019a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:51.805132 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1226/2430] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:42:51 2022] DELETE /identity/v3/projects/09797e92acc04ffdaf58f6ac35fa105f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:51.810710 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13581cd5-a8f7-4946-93cc-0639c35cb4b7 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:51.811023 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13581cd5-a8f7-4946-93cc-0639c35cb4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:51.811294 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13581cd5-a8f7-4946-93cc-0639c35cb4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:51.830848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-13581cd5-a8f7-4946-93cc-0639c35cb4b7 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:51.832044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-13581cd5-a8f7-4946-93cc-0639c35cb4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:51.891941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1205/2431] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:51.901336 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832284f-9d49-48db-88b6-ede9c5aa6c40 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:51.901602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832284f-9d49-48db-88b6-ede9c5aa6c40 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:51.901885 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832284f-9d49-48db-88b6-ede9c5aa6c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:51.923049 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-b832284f-9d49-48db-88b6-ede9c5aa6c40 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:51.924274 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b832284f-9d49-48db-88b6-ede9c5aa6c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:51.933653 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1227/2432] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:42:51.938167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13581cd5-a8f7-4946-93cc-0639c35cb4b7 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:51.948107 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13581cd5-a8f7-4946-93cc-0639c35cb4b7 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:51.954251 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688162f8-ecaf-4181-a1e7-d92a2a067d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:51.955512 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688162f8-ecaf-4181-a1e7-d92a2a067d66 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:51.955826 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688162f8-ecaf-4181-a1e7-d92a2a067d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:51.956099 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688162f8-ecaf-4181-a1e7-d92a2a067d66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:51.956866 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688162f8-ecaf-4181-a1e7-d92a2a067d66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:42:51.957226 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688162f8-ecaf-4181-a1e7-d92a2a067d66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:51.958880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688162f8-ecaf-4181-a1e7-d92a2a067d66 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:51.972031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1206/2433] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:42:51.976597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b832284f-9d49-48db-88b6-ede9c5aa6c40 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:51.986561 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b832284f-9d49-48db-88b6-ede9c5aa6c40 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:51.991940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b91fa27-43b4-4125-894a-20bdf01140d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:51.992625 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91fa27-43b4-4125-894a-20bdf01140d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:51.993002 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91fa27-43b4-4125-894a-20bdf01140d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:51.993342 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91fa27-43b4-4125-894a-20bdf01140d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:51.994082 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b91fa27-43b4-4125-894a-20bdf01140d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:51.996297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b91fa27-43b4-4125-894a-20bdf01140d6 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:52.013149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:52.037474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1228/2434] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:42:51 2022] POST /identity/v3/projects => generated 391 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:42:52.038648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688162f8-ecaf-4181-a1e7-d92a2a067d66 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:52.050100 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688162f8-ecaf-4181-a1e7-d92a2a067d66 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:52.108925 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7a9cf-7e1d-4629-9ad2-8828567483d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:52.109281 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7a9cf-7e1d-4629-9ad2-8828567483d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:52.109566 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7a9cf-7e1d-4629-9ad2-8828567483d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:52.109840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7a9cf-7e1d-4629-9ad2-8828567483d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:52.110500 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7a9cf-7e1d-4629-9ad2-8828567483d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:52.112435 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7a9cf-7e1d-4629-9ad2-8828567483d6 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:52.138104 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1207/2435] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:42:52 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:42:52.139299 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b91fa27-43b4-4125-894a-20bdf01140d6 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:52.163205 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b91fa27-43b4-4125-894a-20bdf01140d6 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:52.170469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddb1be-67de-4103-a64b-4fe677d984f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:52.171170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddb1be-67de-4103-a64b-4fe677d984f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:52.171514 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddb1be-67de-4103-a64b-4fe677d984f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:52.171855 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddb1be-67de-4103-a64b-4fe677d984f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:52.172591 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddb1be-67de-4103-a64b-4fe677d984f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:52.174384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddb1be-67de-4103-a64b-4fe677d984f2 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:52.190937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1229/2436] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:42:52.192568 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7a9cf-7e1d-4629-9ad2-8828567483d6 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:52.194966 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddb1be-67de-4103-a64b-4fe677d984f2 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:52.196588 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-b6ddb1be-67de-4103-a64b-4fe677d984f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:52.203039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7a9cf-7e1d-4629-9ad2-8828567483d6 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:52.207763 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2723b7f-324a-4733-8948-bd8c2d88e82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:52.208443 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2723b7f-324a-4733-8948-bd8c2d88e82a admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:52.208790 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2723b7f-324a-4733-8948-bd8c2d88e82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:52.209166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2723b7f-324a-4733-8948-bd8c2d88e82a admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:52.209949 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2723b7f-324a-4733-8948-bd8c2d88e82a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:52.211624 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2723b7f-324a-4733-8948-bd8c2d88e82a admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:52.223624 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1208/2437] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:42:52.227773 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2723b7f-324a-4733-8948-bd8c2d88e82a admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:52.238035 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2723b7f-324a-4733-8948-bd8c2d88e82a admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:52.242964 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527d00e-6d58-4a93-94bb-b45c55eb5296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:52.243622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527d00e-6d58-4a93-94bb-b45c55eb5296 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:52.243923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527d00e-6d58-4a93-94bb-b45c55eb5296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:52.244181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527d00e-6d58-4a93-94bb-b45c55eb5296 admin admin] PATH_INFO: `/v3/projects/4549252e2321469e9bfee3e3cd7f3521/users/86e65bc229c54bc9ba985814f7e7a138/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:52.257855 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddb1be-67de-4103-a64b-4fe677d984f2 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:52.259415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e527d00e-6d58-4a93-94bb-b45c55eb5296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4549252e2321469e9bfee3e3cd7f3521, user_id=86e65bc229c54bc9ba985814f7e7a138, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:52.261629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a7c0a-1f50-41a5-9f96-f20a9ae693e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:52.262118 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e527d00e-6d58-4a93-94bb-b45c55eb5296 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:52.262298 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a7c0a-1f50-41a5-9f96-f20a9ae693e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:52.262590 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a7c0a-1f50-41a5-9f96-f20a9ae693e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:52.262912 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a7c0a-1f50-41a5-9f96-f20a9ae693e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:52.265084 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-3a3a7c0a-1f50-41a5-9f96-f20a9ae693e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:52.274870 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1209/2438] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:42:52 2022] PUT /identity/v3/projects/4549252e2321469e9bfee3e3cd7f3521/users/86e65bc229c54bc9ba985814f7e7a138/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:42:52.284013 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a7c0a-1f50-41a5-9f96-f20a9ae693e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:52.284013 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a7c0a-1f50-41a5-9f96-f20a9ae693e4 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:52.286530 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbee71e-49b0-4dff-b3fa-f8760c0f9297 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:52.286886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbee71e-49b0-4dff-b3fa-f8760c0f9297 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:52.287170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbee71e-49b0-4dff-b3fa-f8760c0f9297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:52.291091 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1230/2439] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:42:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:42:52.308532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:52.318933 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-0fbee71e-49b0-4dff-b3fa-f8760c0f9297 None None] MFA Rules not processed for user `86e65bc229c54bc9ba985814f7e7a138`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:52.320174 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0fbee71e-49b0-4dff-b3fa-f8760c0f9297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:52.390986 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1210/2440] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:42:52.402054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a7c0a-1f50-41a5-9f96-f20a9ae693e4 service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:52.403691 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-3a3a7c0a-1f50-41a5-9f96-f20a9ae693e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:52.468474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a7c0a-1f50-41a5-9f96-f20a9ae693e4 service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:52.534405 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978d96e-1fb8-4471-b434-89d2b6d954ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:52.535124 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978d96e-1fb8-4471-b434-89d2b6d954ef service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:52.535472 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978d96e-1fb8-4471-b434-89d2b6d954ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:52.535817 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978d96e-1fb8-4471-b434-89d2b6d954ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:52.545869 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978d96e-1fb8-4471-b434-89d2b6d954ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:52.547394 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978d96e-1fb8-4471-b434-89d2b6d954ef service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:52.558939 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1231/2441] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:42:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:42:55.803160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbee71e-49b0-4dff-b3fa-f8760c0f9297 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:55.804723 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0fbee71e-49b0-4dff-b3fa-f8760c0f9297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:55.870769 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbee71e-49b0-4dff-b3fa-f8760c0f9297 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:55.875234 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb5d24-84bb-4ac7-8e88-fd882e9f596a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:55.875868 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb5d24-84bb-4ac7-8e88-fd882e9f596a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:55.876142 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb5d24-84bb-4ac7-8e88-fd882e9f596a admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:55.876412 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb5d24-84bb-4ac7-8e88-fd882e9f596a admin admin] PATH_INFO: `/v3/users/140059c5b30c426ab6d7f1236eb5f684` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:55.877566 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb5d24-84bb-4ac7-8e88-fd882e9f596a admin admin] RBAC: Authorizing `identity:delete_user(user_id=140059c5b30c426ab6d7f1236eb5f684)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:55.879508 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb5d24-84bb-4ac7-8e88-fd882e9f596a admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:55.925618 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-22eb5d24-84bb-4ac7-8e88-fd882e9f596a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '140059c5b30c426ab6d7f1236eb5f684'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:55.932479 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-22eb5d24-84bb-4ac7-8e88-fd882e9f596a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '140059c5b30c426ab6d7f1236eb5f684'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:55.939457 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-22eb5d24-84bb-4ac7-8e88-fd882e9f596a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '140059c5b30c426ab6d7f1236eb5f684'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:55.951593 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-22eb5d24-84bb-4ac7-8e88-fd882e9f596a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '140059c5b30c426ab6d7f1236eb5f684'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:55.952283 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-22eb5d24-84bb-4ac7-8e88-fd882e9f596a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '140059c5b30c426ab6d7f1236eb5f684'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:55.958288 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1211/2442] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:42:55 2022] DELETE /identity/v3/users/140059c5b30c426ab6d7f1236eb5f684 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:56.131740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978d96e-1fb8-4471-b434-89d2b6d954ef service cinder] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:56.145390 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978d96e-1fb8-4471-b434-89d2b6d954ef service cinder] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:56.180217 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:56.205854 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470af796-0078-48ac-95c4-1022edfc6965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:56.206506 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470af796-0078-48ac-95c4-1022edfc6965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:56.206788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470af796-0078-48ac-95c4-1022edfc6965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:56.207065 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470af796-0078-48ac-95c4-1022edfc6965 admin admin] PATH_INFO: `/v3/projects/ebcc91e5911a4a418118b3775cfe4b76` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:56.208206 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470af796-0078-48ac-95c4-1022edfc6965 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebcc91e5911a4a418118b3775cfe4b76)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:56.210184 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470af796-0078-48ac-95c4-1022edfc6965 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:56.256180 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-470af796-0078-48ac-95c4-1022edfc6965 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebcc91e5911a4a418118b3775cfe4b76', 'request_id': 'req-470af796-0078-48ac-95c4-1022edfc6965'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:56.264167 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-470af796-0078-48ac-95c4-1022edfc6965 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebcc91e5911a4a418118b3775cfe4b76', 'request_id': 'req-470af796-0078-48ac-95c4-1022edfc6965'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:56.269796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-470af796-0078-48ac-95c4-1022edfc6965 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebcc91e5911a4a418118b3775cfe4b76', 'request_id': 'req-470af796-0078-48ac-95c4-1022edfc6965'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:56.280481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-470af796-0078-48ac-95c4-1022edfc6965 admin admin] The token cache is being invalidate because project ebcc91e5911a4a418118b3775cfe4b76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:42:56.280770 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-470af796-0078-48ac-95c4-1022edfc6965 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:56.281438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-470af796-0078-48ac-95c4-1022edfc6965 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:42:56.282947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1232/2443] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:42:56 2022] DELETE /identity/v3/projects/ebcc91e5911a4a418118b3775cfe4b76 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:56.287804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450053e5-5f4c-46ae-bc6d-38aa5ab1db47 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:56.288095 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450053e5-5f4c-46ae-bc6d-38aa5ab1db47 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:56.288378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450053e5-5f4c-46ae-bc6d-38aa5ab1db47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:56.307426 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-450053e5-5f4c-46ae-bc6d-38aa5ab1db47 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:56.308638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-450053e5-5f4c-46ae-bc6d-38aa5ab1db47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:56.322572 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:56.377198 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1212/2444] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:56.386991 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d1bf4-3ca5-41ce-b496-e90dbf875d99 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:56.387688 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d1bf4-3ca5-41ce-b496-e90dbf875d99 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:56.387946 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d1bf4-3ca5-41ce-b496-e90dbf875d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:56.417338 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-f29d1bf4-3ca5-41ce-b496-e90dbf875d99 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:56.418548 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f29d1bf4-3ca5-41ce-b496-e90dbf875d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:56.426616 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1233/2445] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:56.430916 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450053e5-5f4c-46ae-bc6d-38aa5ab1db47 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:56.445513 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450053e5-5f4c-46ae-bc6d-38aa5ab1db47 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:56.450167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9daec8-eb04-4e6f-855c-a58455ca0820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:56.451433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9daec8-eb04-4e6f-855c-a58455ca0820 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:56.451714 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9daec8-eb04-4e6f-855c-a58455ca0820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:56.451984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9daec8-eb04-4e6f-855c-a58455ca0820 admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:56.452808 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9daec8-eb04-4e6f-855c-a58455ca0820 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:42:56.453037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9daec8-eb04-4e6f-855c-a58455ca0820 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:56.454682 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9daec8-eb04-4e6f-855c-a58455ca0820 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:56.466980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1213/2446] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:42:56.471623 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d1bf4-3ca5-41ce-b496-e90dbf875d99 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:56.481202 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d1bf4-3ca5-41ce-b496-e90dbf875d99 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:56.486093 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc0d77-987c-412c-adb0-7a47bfa9565f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:56.486707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc0d77-987c-412c-adb0-7a47bfa9565f admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:56.486992 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc0d77-987c-412c-adb0-7a47bfa9565f admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:56.487266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc0d77-987c-412c-adb0-7a47bfa9565f admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:56.487937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc0d77-987c-412c-adb0-7a47bfa9565f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:56.489877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc0d77-987c-412c-adb0-7a47bfa9565f admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:56.517397 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1234/2447] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:42:56 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:42:56.521804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9daec8-eb04-4e6f-855c-a58455ca0820 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:56.531849 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9daec8-eb04-4e6f-855c-a58455ca0820 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:56.587672 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af760082-05e2-48c1-9e92-4d9e4246b1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:56.588274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af760082-05e2-48c1-9e92-4d9e4246b1c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:56.588554 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af760082-05e2-48c1-9e92-4d9e4246b1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:56.588826 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af760082-05e2-48c1-9e92-4d9e4246b1c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:56.589501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af760082-05e2-48c1-9e92-4d9e4246b1c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:56.591431 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af760082-05e2-48c1-9e92-4d9e4246b1c5 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:56.613137 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1214/2448] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:42:56 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:42:56.616970 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc0d77-987c-412c-adb0-7a47bfa9565f admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:56.629189 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc0d77-987c-412c-adb0-7a47bfa9565f admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:56.634115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea2c53-62e0-4545-8ca0-460e665ad0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:56.634740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea2c53-62e0-4545-8ca0-460e665ad0ac admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:56.635111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea2c53-62e0-4545-8ca0-460e665ad0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:56.635391 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea2c53-62e0-4545-8ca0-460e665ad0ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:56.636084 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea2c53-62e0-4545-8ca0-460e665ad0ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:56.637777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea2c53-62e0-4545-8ca0-460e665ad0ac admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:56.648532 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1235/2449] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:42:56.652719 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af760082-05e2-48c1-9e92-4d9e4246b1c5 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:56.663052 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af760082-05e2-48c1-9e92-4d9e4246b1c5 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:56.668063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4408c85-bd9c-4509-88e5-a1be680216ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:56.668942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4408c85-bd9c-4509-88e5-a1be680216ba admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:56.669232 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4408c85-bd9c-4509-88e5-a1be680216ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:56.670048 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4408c85-bd9c-4509-88e5-a1be680216ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:56.670778 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4408c85-bd9c-4509-88e5-a1be680216ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:56.672400 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4408c85-bd9c-4509-88e5-a1be680216ba admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:56.683696 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1215/2450] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:42:56.688464 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea2c53-62e0-4545-8ca0-460e665ad0ac admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:56.698865 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea2c53-62e0-4545-8ca0-460e665ad0ac admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:56.704277 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecba27-73c2-4591-a4a2-6c255368846e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:56.704986 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecba27-73c2-4591-a4a2-6c255368846e admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:56.705350 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecba27-73c2-4591-a4a2-6c255368846e admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:56.705632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecba27-73c2-4591-a4a2-6c255368846e admin admin] PATH_INFO: `/v3/projects/1ea50f049f1f43018a41f9891b6c1993/users/2b7b17f196c44117959c829f965c8fd3/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:56.720813 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acecba27-73c2-4591-a4a2-6c255368846e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ea50f049f1f43018a41f9891b6c1993, user_id=2b7b17f196c44117959c829f965c8fd3, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:56.723516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acecba27-73c2-4591-a4a2-6c255368846e admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:56.742747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1236/2451] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:42:56 2022] PUT /identity/v3/projects/1ea50f049f1f43018a41f9891b6c1993/users/2b7b17f196c44117959c829f965c8fd3/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:56.754149 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68de5bb-2bb3-449c-bd99-e1b276e76a22 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:56.754441 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68de5bb-2bb3-449c-bd99-e1b276e76a22 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:56.754725 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68de5bb-2bb3-449c-bd99-e1b276e76a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:56.783335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-c68de5bb-2bb3-449c-bd99-e1b276e76a22 None None] MFA Rules not processed for user `2b7b17f196c44117959c829f965c8fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:56.784552 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c68de5bb-2bb3-449c-bd99-e1b276e76a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:56.849241 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1216/2452] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:56.853692 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecba27-73c2-4591-a4a2-6c255368846e admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:56.915303 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecba27-73c2-4591-a4a2-6c255368846e admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:56.920446 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb852b-b1ae-4fdd-9b92-1eb16d471f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:56.921082 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb852b-b1ae-4fdd-9b92-1eb16d471f6e admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:56.921363 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb852b-b1ae-4fdd-9b92-1eb16d471f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:56.921636 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb852b-b1ae-4fdd-9b92-1eb16d471f6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:56.922282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb852b-b1ae-4fdd-9b92-1eb16d471f6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:56.924166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb852b-b1ae-4fdd-9b92-1eb16d471f6e admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:56.929354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:56.952785 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1237/2453] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:42:56 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:42:56.956984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68de5bb-2bb3-449c-bd99-e1b276e76a22 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:56.968033 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68de5bb-2bb3-449c-bd99-e1b276e76a22 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:56.977392 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:57.025926 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9eabf-de92-4b30-9f05-a0b292fee699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:57.026543 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9eabf-de92-4b30-9f05-a0b292fee699 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:57.026826 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9eabf-de92-4b30-9f05-a0b292fee699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:57.027100 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9eabf-de92-4b30-9f05-a0b292fee699 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:57.027759 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9eabf-de92-4b30-9f05-a0b292fee699 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:57.030259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9eabf-de92-4b30-9f05-a0b292fee699 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:57.052130 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1217/2454] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:42:56 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:42:57.055923 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb852b-b1ae-4fdd-9b92-1eb16d471f6e admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:57.066390 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb852b-b1ae-4fdd-9b92-1eb16d471f6e admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:57.071050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8f669-cab3-4de0-82ab-4f4540ba45c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:57.071667 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8f669-cab3-4de0-82ab-4f4540ba45c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:57.071949 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8f669-cab3-4de0-82ab-4f4540ba45c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:57.072223 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8f669-cab3-4de0-82ab-4f4540ba45c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:57.072944 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8f669-cab3-4de0-82ab-4f4540ba45c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:57.074588 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8f669-cab3-4de0-82ab-4f4540ba45c1 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:57.084743 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1238/2455] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:42:57.088998 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9eabf-de92-4b30-9f05-a0b292fee699 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:57.101333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9eabf-de92-4b30-9f05-a0b292fee699 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:57.106173 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8cce7-877f-4730-8466-69442b22475e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:57.107598 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8cce7-877f-4730-8466-69442b22475e admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:57.107896 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8cce7-877f-4730-8466-69442b22475e admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:57.108183 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8cce7-877f-4730-8466-69442b22475e admin admin] PATH_INFO: `/v3/projects/e3c2c794a6be4f26852872e34ab64341/users/2a20f1cbc12e40b2b49aef6dbb338709/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:57.124707 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8cce7-877f-4730-8466-69442b22475e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3c2c794a6be4f26852872e34ab64341, user_id=2a20f1cbc12e40b2b49aef6dbb338709, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:57.127665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8cce7-877f-4730-8466-69442b22475e admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:57.141973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1218/2456] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:42:57 2022] PUT /identity/v3/projects/e3c2c794a6be4f26852872e34ab64341/users/2a20f1cbc12e40b2b49aef6dbb338709/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:57.145872 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8f669-cab3-4de0-82ab-4f4540ba45c1 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:57.157569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8f669-cab3-4de0-82ab-4f4540ba45c1 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:57.219105 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfeee93-2984-4038-825c-fdf036c1e28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:57.220042 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeee93-2984-4038-825c-fdf036c1e28e admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:57.220327 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeee93-2984-4038-825c-fdf036c1e28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:57.220603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeee93-2984-4038-825c-fdf036c1e28e admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:57.221334 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfeee93-2984-4038-825c-fdf036c1e28e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:57.222946 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfeee93-2984-4038-825c-fdf036c1e28e admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:57.238110 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1239/2457] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:42:57.239303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8cce7-877f-4730-8466-69442b22475e admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:57.251413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8cce7-877f-4730-8466-69442b22475e admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:57.256533 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2bd621-5dad-42ae-9431-d248fa5afef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:57.257258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bd621-5dad-42ae-9431-d248fa5afef3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:57.257695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bd621-5dad-42ae-9431-d248fa5afef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:57.257812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bd621-5dad-42ae-9431-d248fa5afef3 admin admin] PATH_INFO: `/v3/projects/e3c2c794a6be4f26852872e34ab64341/users/2a20f1cbc12e40b2b49aef6dbb338709/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:57.262524 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2bd621-5dad-42ae-9431-d248fa5afef3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3c2c794a6be4f26852872e34ab64341, user_id=2a20f1cbc12e40b2b49aef6dbb338709, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:57.266461 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2bd621-5dad-42ae-9431-d248fa5afef3 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:57.280789 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1219/2458] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:42:57 2022] PUT /identity/v3/projects/e3c2c794a6be4f26852872e34ab64341/users/2a20f1cbc12e40b2b49aef6dbb338709/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:42:57.292865 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7439d-d24b-4f24-b4e9-28d3afad6729 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:57.293255 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7439d-d24b-4f24-b4e9-28d3afad6729 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:57.293463 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7439d-d24b-4f24-b4e9-28d3afad6729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:57.323427 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-8be7439d-d24b-4f24-b4e9-28d3afad6729 None None] MFA Rules not processed for user `2a20f1cbc12e40b2b49aef6dbb338709`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:42:57.325206 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-8be7439d-d24b-4f24-b4e9-28d3afad6729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:57.411851 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1240/2459] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:42:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:42:57.424172 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2bd621-5dad-42ae-9431-d248fa5afef3 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:57.429110 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-fc2bd621-5dad-42ae-9431-d248fa5afef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:42:57.509169 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2bd621-5dad-42ae-9431-d248fa5afef3 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:57.513030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676aac7e-f1f5-4279-9ce1-b388e18e9ef9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:57.514259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676aac7e-f1f5-4279-9ce1-b388e18e9ef9 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:57.514635 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676aac7e-f1f5-4279-9ce1-b388e18e9ef9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:57.515036 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676aac7e-f1f5-4279-9ce1-b388e18e9ef9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:57.524768 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676aac7e-f1f5-4279-9ce1-b388e18e9ef9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:57.526308 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676aac7e-f1f5-4279-9ce1-b388e18e9ef9 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:57.536563 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1220/2460] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:42:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:42:57.625626 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7439d-d24b-4f24-b4e9-28d3afad6729 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:42:57.636053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7439d-d24b-4f24-b4e9-28d3afad6729 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:42:57.640242 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0bfc9f-656f-4c6f-9d11-2f2afe403a73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:42:57.640859 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0bfc9f-656f-4c6f-9d11-2f2afe403a73 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:42:57.641174 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0bfc9f-656f-4c6f-9d11-2f2afe403a73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:42:57.641576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0bfc9f-656f-4c6f-9d11-2f2afe403a73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:42:57.697583 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:42:57.700252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0bfc9f-656f-4c6f-9d11-2f2afe403a73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:42:57.703444 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0bfc9f-656f-4c6f-9d11-2f2afe403a73 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:42:57.713462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1241/2461] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:42:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:07.532271 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=64,keystone:INSERT=5 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:07.709187 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:09.717260 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676aac7e-f1f5-4279-9ce1-b388e18e9ef9 service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:09.778636 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676aac7e-f1f5-4279-9ce1-b388e18e9ef9 service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:09.783057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26ca9a-730e-4b71-8fbf-b30e2199d269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:09.783717 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26ca9a-730e-4b71-8fbf-b30e2199d269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:09.784001 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26ca9a-730e-4b71-8fbf-b30e2199d269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:09.784268 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26ca9a-730e-4b71-8fbf-b30e2199d269 admin admin] PATH_INFO: `/v3/users/2b7b17f196c44117959c829f965c8fd3` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:09.785425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26ca9a-730e-4b71-8fbf-b30e2199d269 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b7b17f196c44117959c829f965c8fd3)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:09.787361 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26ca9a-730e-4b71-8fbf-b30e2199d269 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:09.835591 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-ca26ca9a-730e-4b71-8fbf-b30e2199d269 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b7b17f196c44117959c829f965c8fd3'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:09.843608 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-ca26ca9a-730e-4b71-8fbf-b30e2199d269 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b7b17f196c44117959c829f965c8fd3'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:09.851148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-ca26ca9a-730e-4b71-8fbf-b30e2199d269 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b7b17f196c44117959c829f965c8fd3'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:09.867963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-ca26ca9a-730e-4b71-8fbf-b30e2199d269 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b7b17f196c44117959c829f965c8fd3'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:09.868689 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-ca26ca9a-730e-4b71-8fbf-b30e2199d269 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b7b17f196c44117959c829f965c8fd3'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:09.875535 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1221/2462] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:43:09 2022] DELETE /identity/v3/users/2b7b17f196c44117959c829f965c8fd3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:09.880311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c11c70-944a-4e1d-b431-3ab9fcc948d6 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:09.880768 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c11c70-944a-4e1d-b431-3ab9fcc948d6 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:09.880869 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c11c70-944a-4e1d-b431-3ab9fcc948d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:09.910759 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-76c11c70-944a-4e1d-b431-3ab9fcc948d6 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:09.912041 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-76c11c70-944a-4e1d-b431-3ab9fcc948d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:09.977735 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1242/2463] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:43:09.988722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26ca9a-730e-4b71-8fbf-b30e2199d269 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:09.990294 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ca26ca9a-730e-4b71-8fbf-b30e2199d269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:10.055129 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26ca9a-730e-4b71-8fbf-b30e2199d269 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:10.058766 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a3d00-b249-4da2-abb8-754f08987475 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:10.059383 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a3d00-b249-4da2-abb8-754f08987475 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:10.059669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a3d00-b249-4da2-abb8-754f08987475 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:10.060493 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0a3d00-b249-4da2-abb8-754f08987475 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:10.069633 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a3d00-b249-4da2-abb8-754f08987475 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:10.071064 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0a3d00-b249-4da2-abb8-754f08987475 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:10.080483 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1222/2464] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:43:10.354789 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c11c70-944a-4e1d-b431-3ab9fcc948d6 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:10.356294 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-76c11c70-944a-4e1d-b431-3ab9fcc948d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:10.369470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c11c70-944a-4e1d-b431-3ab9fcc948d6 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:10.374744 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110b6bf-9f62-42c0-9c72-107aa929bf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:10.375659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110b6bf-9f62-42c0-9c72-107aa929bf71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:10.375946 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110b6bf-9f62-42c0-9c72-107aa929bf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:10.376224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110b6bf-9f62-42c0-9c72-107aa929bf71 admin admin] PATH_INFO: `/v3/projects/1ea50f049f1f43018a41f9891b6c1993` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:10.377441 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110b6bf-9f62-42c0-9c72-107aa929bf71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ea50f049f1f43018a41f9891b6c1993)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:10.379373 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110b6bf-9f62-42c0-9c72-107aa929bf71 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:10.429597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-7110b6bf-9f62-42c0-9c72-107aa929bf71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ea50f049f1f43018a41f9891b6c1993', 'request_id': 'req-7110b6bf-9f62-42c0-9c72-107aa929bf71'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:10.437308 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-7110b6bf-9f62-42c0-9c72-107aa929bf71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ea50f049f1f43018a41f9891b6c1993', 'request_id': 'req-7110b6bf-9f62-42c0-9c72-107aa929bf71'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:10.443942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-7110b6bf-9f62-42c0-9c72-107aa929bf71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ea50f049f1f43018a41f9891b6c1993', 'request_id': 'req-7110b6bf-9f62-42c0-9c72-107aa929bf71'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:10.455017 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-7110b6bf-9f62-42c0-9c72-107aa929bf71 admin admin] The token cache is being invalidate because project 1ea50f049f1f43018a41f9891b6c1993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:43:10.455306 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-7110b6bf-9f62-42c0-9c72-107aa929bf71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:10.455964 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-7110b6bf-9f62-42c0-9c72-107aa929bf71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:10.457570 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1243/2465] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:43:10 2022] DELETE /identity/v3/projects/1ea50f049f1f43018a41f9891b6c1993 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:10.461636 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a3d00-b249-4da2-abb8-754f08987475 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:10.472201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0a3d00-b249-4da2-abb8-754f08987475 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:10.491045 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:10.533044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d419ef8-1957-400b-b02f-96f842286180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:10.533697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d419ef8-1957-400b-b02f-96f842286180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:10.533979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d419ef8-1957-400b-b02f-96f842286180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:10.534261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d419ef8-1957-400b-b02f-96f842286180 admin admin] PATH_INFO: `/v3/users/2a20f1cbc12e40b2b49aef6dbb338709` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:10.535420 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d419ef8-1957-400b-b02f-96f842286180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a20f1cbc12e40b2b49aef6dbb338709)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:10.538629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d419ef8-1957-400b-b02f-96f842286180 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:10.594551 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-1d419ef8-1957-400b-b02f-96f842286180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a20f1cbc12e40b2b49aef6dbb338709'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:10.603545 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-1d419ef8-1957-400b-b02f-96f842286180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a20f1cbc12e40b2b49aef6dbb338709'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:10.611239 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-1d419ef8-1957-400b-b02f-96f842286180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a20f1cbc12e40b2b49aef6dbb338709'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:10.623415 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-1d419ef8-1957-400b-b02f-96f842286180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a20f1cbc12e40b2b49aef6dbb338709'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:10.624104 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-1d419ef8-1957-400b-b02f-96f842286180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a20f1cbc12e40b2b49aef6dbb338709'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:10.630094 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1223/2466] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:43:10 2022] DELETE /identity/v3/users/2a20f1cbc12e40b2b49aef6dbb338709 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:43:10.922156 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110b6bf-9f62-42c0-9c72-107aa929bf71 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:10.923595 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7110b6bf-9f62-42c0-9c72-107aa929bf71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:10.939940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110b6bf-9f62-42c0-9c72-107aa929bf71 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:11.011940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8503c5-650e-4de8-a758-3d94b34f22a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:11.012602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8503c5-650e-4de8-a758-3d94b34f22a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:11.013428 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8503c5-650e-4de8-a758-3d94b34f22a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:11.013709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8503c5-650e-4de8-a758-3d94b34f22a0 admin admin] PATH_INFO: `/v3/projects/e3c2c794a6be4f26852872e34ab64341` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:11.015370 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8503c5-650e-4de8-a758-3d94b34f22a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3c2c794a6be4f26852872e34ab64341)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:11.018420 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8503c5-650e-4de8-a758-3d94b34f22a0 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:11.052351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:11.069715 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6d8503c5-650e-4de8-a758-3d94b34f22a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3c2c794a6be4f26852872e34ab64341', 'request_id': 'req-6d8503c5-650e-4de8-a758-3d94b34f22a0'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:11.077509 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6d8503c5-650e-4de8-a758-3d94b34f22a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3c2c794a6be4f26852872e34ab64341', 'request_id': 'req-6d8503c5-650e-4de8-a758-3d94b34f22a0'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:11.083462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6d8503c5-650e-4de8-a758-3d94b34f22a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3c2c794a6be4f26852872e34ab64341', 'request_id': 'req-6d8503c5-650e-4de8-a758-3d94b34f22a0'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:11.095168 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6d8503c5-650e-4de8-a758-3d94b34f22a0 admin admin] The token cache is being invalidate because project e3c2c794a6be4f26852872e34ab64341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:43:11.095479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6d8503c5-650e-4de8-a758-3d94b34f22a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:11.096107 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6d8503c5-650e-4de8-a758-3d94b34f22a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:11.097653 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1244/2467] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:43:10 2022] DELETE /identity/v3/projects/e3c2c794a6be4f26852872e34ab64341 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:11.103500 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dfe0f-bd96-4a28-9cab-9c4967eb8416 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:11.103991 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dfe0f-bd96-4a28-9cab-9c4967eb8416 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:11.104091 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95dfe0f-bd96-4a28-9cab-9c4967eb8416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:11.133682 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-f95dfe0f-bd96-4a28-9cab-9c4967eb8416 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:11.134924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-f95dfe0f-bd96-4a28-9cab-9c4967eb8416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:11.201912 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1224/2468] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:11.209784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274be8c-4e84-48da-985d-bcd5aeac2bc2 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:11.210089 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274be8c-4e84-48da-985d-bcd5aeac2bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:11.210351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274be8c-4e84-48da-985d-bcd5aeac2bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:11.230938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-5274be8c-4e84-48da-985d-bcd5aeac2bc2 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:11.232239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-5274be8c-4e84-48da-985d-bcd5aeac2bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:11.240354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1245/2469] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:11.244758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dfe0f-bd96-4a28-9cab-9c4967eb8416 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:11.255053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95dfe0f-bd96-4a28-9cab-9c4967eb8416 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:11.260013 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834310f-bd08-4211-9271-82b9bf2cd9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:11.260755 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834310f-bd08-4211-9271-82b9bf2cd9ad admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:11.260916 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834310f-bd08-4211-9271-82b9bf2cd9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:11.261213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834310f-bd08-4211-9271-82b9bf2cd9ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:11.261907 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834310f-bd08-4211-9271-82b9bf2cd9ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:43:11.262212 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834310f-bd08-4211-9271-82b9bf2cd9ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:11.264400 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9834310f-bd08-4211-9271-82b9bf2cd9ad admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:11.276623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1225/2470] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:43:11.281530 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5274be8c-4e84-48da-985d-bcd5aeac2bc2 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:11.293798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5274be8c-4e84-48da-985d-bcd5aeac2bc2 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:11.298612 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888f474-00a7-4c6a-b727-8c4097478d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:11.299232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888f474-00a7-4c6a-b727-8c4097478d4b admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:11.299517 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888f474-00a7-4c6a-b727-8c4097478d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:11.299797 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888f474-00a7-4c6a-b727-8c4097478d4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:11.300468 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888f474-00a7-4c6a-b727-8c4097478d4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:11.302408 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888f474-00a7-4c6a-b727-8c4097478d4b admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:11.333490 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1246/2471] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:43:11 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:11.337781 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834310f-bd08-4211-9271-82b9bf2cd9ad admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:11.347999 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834310f-bd08-4211-9271-82b9bf2cd9ad admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:11.373771 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:11.405666 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127dd4b3-3cff-443f-93c9-dd485e9334c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:11.406216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127dd4b3-3cff-443f-93c9-dd485e9334c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:11.406529 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127dd4b3-3cff-443f-93c9-dd485e9334c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:11.406835 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127dd4b3-3cff-443f-93c9-dd485e9334c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:11.407491 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127dd4b3-3cff-443f-93c9-dd485e9334c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:11.409480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127dd4b3-3cff-443f-93c9-dd485e9334c4 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:11.429847 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1226/2472] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:43:11 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:43:11.434134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888f474-00a7-4c6a-b727-8c4097478d4b admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:11.444787 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888f474-00a7-4c6a-b727-8c4097478d4b admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:11.449355 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb7e31-4ed5-4f26-aaa0-11add77ab83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:11.449952 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb7e31-4ed5-4f26-aaa0-11add77ab83a admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:11.450257 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb7e31-4ed5-4f26-aaa0-11add77ab83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:11.451141 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb7e31-4ed5-4f26-aaa0-11add77ab83a admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:11.451828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb7e31-4ed5-4f26-aaa0-11add77ab83a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:11.453448 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb7e31-4ed5-4f26-aaa0-11add77ab83a admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:11.463213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1247/2473] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:43:11.466992 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127dd4b3-3cff-443f-93c9-dd485e9334c4 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:11.477254 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127dd4b3-3cff-443f-93c9-dd485e9334c4 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:11.481920 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ebffb-aa59-43b7-a7ac-fd2e758ccea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:11.482525 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ebffb-aa59-43b7-a7ac-fd2e758ccea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:11.482810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ebffb-aa59-43b7-a7ac-fd2e758ccea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:11.483089 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ebffb-aa59-43b7-a7ac-fd2e758ccea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:11.483777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ebffb-aa59-43b7-a7ac-fd2e758ccea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:11.485412 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ebffb-aa59-43b7-a7ac-fd2e758ccea5 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:11.495781 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1227/2474] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:11.500213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb7e31-4ed5-4f26-aaa0-11add77ab83a admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:11.509954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb7e31-4ed5-4f26-aaa0-11add77ab83a admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:11.515294 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f40648-eacb-4668-bde6-3a296b8d1e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:11.515956 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f40648-eacb-4668-bde6-3a296b8d1e02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:11.516242 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f40648-eacb-4668-bde6-3a296b8d1e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:11.516555 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f40648-eacb-4668-bde6-3a296b8d1e02 admin admin] PATH_INFO: `/v3/projects/10305a8deb1940bb89430348015a1841/users/ab8684bfd1e742c494f257e1a48bb02f/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:11.530809 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f40648-eacb-4668-bde6-3a296b8d1e02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10305a8deb1940bb89430348015a1841, user_id=ab8684bfd1e742c494f257e1a48bb02f, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:11.533539 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f40648-eacb-4668-bde6-3a296b8d1e02 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:11.546514 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1248/2475] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:11 2022] PUT /identity/v3/projects/10305a8deb1940bb89430348015a1841/users/ab8684bfd1e742c494f257e1a48bb02f/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:11.556623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7bc98-2b47-4ace-8274-ff1745eed364 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:11.556944 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7bc98-2b47-4ace-8274-ff1745eed364 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:11.557262 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7bc98-2b47-4ace-8274-ff1745eed364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:11.584993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-32f7bc98-2b47-4ace-8274-ff1745eed364 None None] MFA Rules not processed for user `ab8684bfd1e742c494f257e1a48bb02f`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:11.586208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-32f7bc98-2b47-4ace-8274-ff1745eed364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:11.649378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1228/2476] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:11 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:43:11.653966 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f40648-eacb-4668-bde6-3a296b8d1e02 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:11.715092 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f40648-eacb-4668-bde6-3a296b8d1e02 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:11.719690 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e0274-b3d8-49a5-b5b2-d165179af0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:11.720559 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e0274-b3d8-49a5-b5b2-d165179af0b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:11.720834 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e0274-b3d8-49a5-b5b2-d165179af0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:11.721130 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e0274-b3d8-49a5-b5b2-d165179af0b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:11.721781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e0274-b3d8-49a5-b5b2-d165179af0b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:11.723692 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e0274-b3d8-49a5-b5b2-d165179af0b3 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:11.751117 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1249/2477] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:43:11 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:43:11.755115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7bc98-2b47-4ace-8274-ff1745eed364 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:11.765582 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7bc98-2b47-4ace-8274-ff1745eed364 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:11.821027 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f528bd2-1e68-408f-a7be-aec0418359ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:11.821635 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f528bd2-1e68-408f-a7be-aec0418359ce admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:11.821917 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f528bd2-1e68-408f-a7be-aec0418359ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:11.822191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f528bd2-1e68-408f-a7be-aec0418359ce admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:11.822854 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f528bd2-1e68-408f-a7be-aec0418359ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:11.824768 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f528bd2-1e68-408f-a7be-aec0418359ce admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:11.845370 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1229/2478] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:43:11 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:43:11.849271 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e0274-b3d8-49a5-b5b2-d165179af0b3 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:11.859866 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e0274-b3d8-49a5-b5b2-d165179af0b3 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:11.864425 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc68f85-4e96-4592-8364-fd573901f59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:11.865060 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc68f85-4e96-4592-8364-fd573901f59e admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:11.865341 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc68f85-4e96-4592-8364-fd573901f59e admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:11.865613 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc68f85-4e96-4592-8364-fd573901f59e admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:11.866294 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc68f85-4e96-4592-8364-fd573901f59e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:11.867885 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc68f85-4e96-4592-8364-fd573901f59e admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:11.879627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1250/2479] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:11.884193 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f528bd2-1e68-408f-a7be-aec0418359ce admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:11.895293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f528bd2-1e68-408f-a7be-aec0418359ce admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:11.900047 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d0e48f-b601-4fdd-b078-c52fa951225c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:11.900717 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d0e48f-b601-4fdd-b078-c52fa951225c admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:11.901028 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d0e48f-b601-4fdd-b078-c52fa951225c admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:11.901305 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d0e48f-b601-4fdd-b078-c52fa951225c admin admin] PATH_INFO: `/v3/projects/95e45e7c145f474f803580fdc168fc1e/users/94ef13aa52e74c5bbfc5e5b90043579f/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:11.916561 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d0e48f-b601-4fdd-b078-c52fa951225c admin admin] RBAC: Authorizing `identity:create_grant(project_id=95e45e7c145f474f803580fdc168fc1e, user_id=94ef13aa52e74c5bbfc5e5b90043579f, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:11.919922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d0e48f-b601-4fdd-b078-c52fa951225c admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:11.933292 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1230/2480] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:11 2022] PUT /identity/v3/projects/95e45e7c145f474f803580fdc168fc1e/users/94ef13aa52e74c5bbfc5e5b90043579f/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:43:11.937429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc68f85-4e96-4592-8364-fd573901f59e admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:11.952042 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc68f85-4e96-4592-8364-fd573901f59e admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:11.972080 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:12.012677 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2006b6-08ef-4d61-9d87-258bac2723a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:12.013334 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2006b6-08ef-4d61-9d87-258bac2723a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:12.013617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2006b6-08ef-4d61-9d87-258bac2723a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:12.013917 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2006b6-08ef-4d61-9d87-258bac2723a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:12.014605 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2006b6-08ef-4d61-9d87-258bac2723a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:12.016200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2006b6-08ef-4d61-9d87-258bac2723a5 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:12.030224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1251/2481] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:12.031180 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d0e48f-b601-4fdd-b078-c52fa951225c admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:12.039108 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:12.043564 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d0e48f-b601-4fdd-b078-c52fa951225c admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:12.049585 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31404023-8ee5-422d-a519-af82d825abe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:12.050273 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31404023-8ee5-422d-a519-af82d825abe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:12.050554 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31404023-8ee5-422d-a519-af82d825abe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:12.050867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31404023-8ee5-422d-a519-af82d825abe6 admin admin] PATH_INFO: `/v3/projects/95e45e7c145f474f803580fdc168fc1e/users/94ef13aa52e74c5bbfc5e5b90043579f/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:12.052805 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31404023-8ee5-422d-a519-af82d825abe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95e45e7c145f474f803580fdc168fc1e, user_id=94ef13aa52e74c5bbfc5e5b90043579f, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:12.055497 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31404023-8ee5-422d-a519-af82d825abe6 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:12.066000 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1231/2482] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:12 2022] PUT /identity/v3/projects/95e45e7c145f474f803580fdc168fc1e/users/94ef13aa52e74c5bbfc5e5b90043579f/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:12.077279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c1479-8e93-4c62-b9c5-b065c6020b9f None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:12.077670 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c1479-8e93-4c62-b9c5-b065c6020b9f None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:12.077995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c1479-8e93-4c62-b9c5-b065c6020b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:12.108610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-7a4c1479-8e93-4c62-b9c5-b065c6020b9f None None] MFA Rules not processed for user `94ef13aa52e74c5bbfc5e5b90043579f`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:12.109860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7a4c1479-8e93-4c62-b9c5-b065c6020b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:12.181993 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1252/2483] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:12 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:12.193950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31404023-8ee5-422d-a519-af82d825abe6 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:12.195638 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-31404023-8ee5-422d-a519-af82d825abe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:12.255083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31404023-8ee5-422d-a519-af82d825abe6 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:12.259584 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f772432-a684-4bca-aa1c-4c7de430209e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:12.260217 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f772432-a684-4bca-aa1c-4c7de430209e service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:12.260503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f772432-a684-4bca-aa1c-4c7de430209e service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:12.260781 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f772432-a684-4bca-aa1c-4c7de430209e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:12.271024 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f772432-a684-4bca-aa1c-4c7de430209e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:12.272516 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f772432-a684-4bca-aa1c-4c7de430209e service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:12.282579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1232/2484] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:12 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:12.364192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c1479-8e93-4c62-b9c5-b065c6020b9f None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:12.374806 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c1479-8e93-4c62-b9c5-b065c6020b9f None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:12.378419 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a8d80-d39c-40df-a6aa-5317b485c373 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:12.379023 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a8d80-d39c-40df-a6aa-5317b485c373 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:12.379300 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a8d80-d39c-40df-a6aa-5317b485c373 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:12.379573 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a8d80-d39c-40df-a6aa-5317b485c373 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:12.434792 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a8d80-d39c-40df-a6aa-5317b485c373 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:12.436743 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a8d80-d39c-40df-a6aa-5317b485c373 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:12.445608 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1253/2485] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:12 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:13.179728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f772432-a684-4bca-aa1c-4c7de430209e service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:13.181167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-2f772432-a684-4bca-aa1c-4c7de430209e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:13.245796 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f772432-a684-4bca-aa1c-4c7de430209e service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:13.250545 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9f923-1ee3-46af-b523-10eb3e2bcead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:13.251641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9f923-1ee3-46af-b523-10eb3e2bcead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:13.252148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9f923-1ee3-46af-b523-10eb3e2bcead admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:13.252650 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9f923-1ee3-46af-b523-10eb3e2bcead admin admin] PATH_INFO: `/v3/users/86e65bc229c54bc9ba985814f7e7a138` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:13.254154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9f923-1ee3-46af-b523-10eb3e2bcead admin admin] RBAC: Authorizing `identity:delete_user(user_id=86e65bc229c54bc9ba985814f7e7a138)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:13.256096 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9f923-1ee3-46af-b523-10eb3e2bcead admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:13.306413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-82c9f923-1ee3-46af-b523-10eb3e2bcead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86e65bc229c54bc9ba985814f7e7a138'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:13.313651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-82c9f923-1ee3-46af-b523-10eb3e2bcead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86e65bc229c54bc9ba985814f7e7a138'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:13.320551 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-82c9f923-1ee3-46af-b523-10eb3e2bcead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86e65bc229c54bc9ba985814f7e7a138'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:13.332647 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-82c9f923-1ee3-46af-b523-10eb3e2bcead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86e65bc229c54bc9ba985814f7e7a138'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:13.333292 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-82c9f923-1ee3-46af-b523-10eb3e2bcead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86e65bc229c54bc9ba985814f7e7a138'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:13.339135 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1233/2486] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:43:13 2022] DELETE /identity/v3/users/86e65bc229c54bc9ba985814f7e7a138 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:13.344301 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1f4d3-fe4d-4ec0-ac58-5c982c5990ab None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:13.344768 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1f4d3-fe4d-4ec0-ac58-5c982c5990ab None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:13.344865 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1f4d3-fe4d-4ec0-ac58-5c982c5990ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:13.363832 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-62e1f4d3-fe4d-4ec0-ac58-5c982c5990ab None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:13.365097 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-62e1f4d3-fe4d-4ec0-ac58-5c982c5990ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:13.405668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:13.426995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1254/2487] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:13.436915 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9f923-1ee3-46af-b523-10eb3e2bcead admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:13.438328 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-82c9f923-1ee3-46af-b523-10eb3e2bcead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:13.452402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:13.496818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9f923-1ee3-46af-b523-10eb3e2bcead admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:13.501057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abdecf4-9cc9-48c5-bd39-b7e94e15473a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:13.501669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdecf4-9cc9-48c5-bd39-b7e94e15473a service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:13.501954 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdecf4-9cc9-48c5-bd39-b7e94e15473a service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:13.502231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdecf4-9cc9-48c5-bd39-b7e94e15473a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:13.511159 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abdecf4-9cc9-48c5-bd39-b7e94e15473a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:13.512597 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abdecf4-9cc9-48c5-bd39-b7e94e15473a service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:13.522905 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1234/2488] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:14.043629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1f4d3-fe4d-4ec0-ac58-5c982c5990ab None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:14.045894 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-62e1f4d3-fe4d-4ec0-ac58-5c982c5990ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:14.061080 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1f4d3-fe4d-4ec0-ac58-5c982c5990ab None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:14.066054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a090334-0dbd-4239-94ff-c3fa7ae33f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:14.066887 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a090334-0dbd-4239-94ff-c3fa7ae33f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:14.067260 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a090334-0dbd-4239-94ff-c3fa7ae33f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:14.067589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a090334-0dbd-4239-94ff-c3fa7ae33f26 admin admin] PATH_INFO: `/v3/projects/4549252e2321469e9bfee3e3cd7f3521` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:14.068822 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a090334-0dbd-4239-94ff-c3fa7ae33f26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4549252e2321469e9bfee3e3cd7f3521)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:14.071020 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a090334-0dbd-4239-94ff-c3fa7ae33f26 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:14.126502 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6a090334-0dbd-4239-94ff-c3fa7ae33f26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4549252e2321469e9bfee3e3cd7f3521', 'request_id': 'req-6a090334-0dbd-4239-94ff-c3fa7ae33f26'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:14.134703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6a090334-0dbd-4239-94ff-c3fa7ae33f26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4549252e2321469e9bfee3e3cd7f3521', 'request_id': 'req-6a090334-0dbd-4239-94ff-c3fa7ae33f26'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:14.140605 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6a090334-0dbd-4239-94ff-c3fa7ae33f26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4549252e2321469e9bfee3e3cd7f3521', 'request_id': 'req-6a090334-0dbd-4239-94ff-c3fa7ae33f26'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:14.152206 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6a090334-0dbd-4239-94ff-c3fa7ae33f26 admin admin] The token cache is being invalidate because project 4549252e2321469e9bfee3e3cd7f3521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:43:14.152493 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6a090334-0dbd-4239-94ff-c3fa7ae33f26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:14.153196 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6a090334-0dbd-4239-94ff-c3fa7ae33f26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:14.155381 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1255/2489] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:43:14 2022] DELETE /identity/v3/projects/4549252e2321469e9bfee3e3cd7f3521 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:14.160069 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4c3d2-80a6-4e3b-baed-2efb808911ed None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:14.160365 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4c3d2-80a6-4e3b-baed-2efb808911ed None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:14.160645 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4c3d2-80a6-4e3b-baed-2efb808911ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:14.181055 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-0ab4c3d2-80a6-4e3b-baed-2efb808911ed None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:14.182262 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0ab4c3d2-80a6-4e3b-baed-2efb808911ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:14.240532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1235/2490] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:14.250114 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07be2d-5fba-48fa-b0bc-c30688250491 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:14.250402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07be2d-5fba-48fa-b0bc-c30688250491 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:14.250683 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07be2d-5fba-48fa-b0bc-c30688250491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:14.270847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-8c07be2d-5fba-48fa-b0bc-c30688250491 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:14.272058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-8c07be2d-5fba-48fa-b0bc-c30688250491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:14.280018 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1256/2491] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:14.284096 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4c3d2-80a6-4e3b-baed-2efb808911ed None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:14.294594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4c3d2-80a6-4e3b-baed-2efb808911ed None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:14.299633 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa4fe4-7f1a-4c22-a597-99f1e826fb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:14.300246 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa4fe4-7f1a-4c22-a597-99f1e826fb32 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:14.300527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa4fe4-7f1a-4c22-a597-99f1e826fb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:14.300800 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa4fe4-7f1a-4c22-a597-99f1e826fb32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:14.301533 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa4fe4-7f1a-4c22-a597-99f1e826fb32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:43:14.301839 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa4fe4-7f1a-4c22-a597-99f1e826fb32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:14.303442 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa4fe4-7f1a-4c22-a597-99f1e826fb32 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:14.316095 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1236/2492] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:43:14.320647 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07be2d-5fba-48fa-b0bc-c30688250491 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:14.330620 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07be2d-5fba-48fa-b0bc-c30688250491 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:14.335414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf97798-57f7-4b7f-9526-37189fb1d5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:14.336021 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf97798-57f7-4b7f-9526-37189fb1d5e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:14.336306 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf97798-57f7-4b7f-9526-37189fb1d5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:14.336583 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf97798-57f7-4b7f-9526-37189fb1d5e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:14.337413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf97798-57f7-4b7f-9526-37189fb1d5e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:14.339315 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf97798-57f7-4b7f-9526-37189fb1d5e0 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:14.366425 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1257/2493] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:43:14 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:14.370871 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa4fe4-7f1a-4c22-a597-99f1e826fb32 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:14.381736 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa4fe4-7f1a-4c22-a597-99f1e826fb32 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:14.437116 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7308d5-aacc-4454-886c-a68e59bc88d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:14.437740 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7308d5-aacc-4454-886c-a68e59bc88d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:14.438024 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7308d5-aacc-4454-886c-a68e59bc88d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:14.438301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7308d5-aacc-4454-886c-a68e59bc88d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:14.438962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7308d5-aacc-4454-886c-a68e59bc88d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:14.440883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7308d5-aacc-4454-886c-a68e59bc88d5 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:14.466012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1237/2494] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:43:14 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:14.468510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf97798-57f7-4b7f-9526-37189fb1d5e0 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:14.484280 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf97798-57f7-4b7f-9526-37189fb1d5e0 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:14.484820 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebaec6b-124e-40e8-ae36-5008990fb248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:14.485479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebaec6b-124e-40e8-ae36-5008990fb248 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:14.485768 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebaec6b-124e-40e8-ae36-5008990fb248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:14.486044 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebaec6b-124e-40e8-ae36-5008990fb248 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:14.486731 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebaec6b-124e-40e8-ae36-5008990fb248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:14.488360 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebaec6b-124e-40e8-ae36-5008990fb248 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:14.499163 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1258/2495] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:14.502835 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7308d5-aacc-4454-886c-a68e59bc88d5 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:14.512791 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7308d5-aacc-4454-886c-a68e59bc88d5 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:14.518005 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4952c9-6366-4dbd-bae4-f2c8c30968c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:14.518610 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4952c9-6366-4dbd-bae4-f2c8c30968c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:14.518893 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4952c9-6366-4dbd-bae4-f2c8c30968c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:14.519194 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4952c9-6366-4dbd-bae4-f2c8c30968c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:14.519884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4952c9-6366-4dbd-bae4-f2c8c30968c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:14.521509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4952c9-6366-4dbd-bae4-f2c8c30968c1 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:14.532248 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1238/2496] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:43:14.536251 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebaec6b-124e-40e8-ae36-5008990fb248 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:14.546778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebaec6b-124e-40e8-ae36-5008990fb248 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:14.551467 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96bde8-5d3d-4692-b77c-1b2e339c6bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:14.552127 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96bde8-5d3d-4692-b77c-1b2e339c6bfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:14.552414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96bde8-5d3d-4692-b77c-1b2e339c6bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:14.552695 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96bde8-5d3d-4692-b77c-1b2e339c6bfc admin admin] PATH_INFO: `/v3/projects/c24a78af0a5d46de89b42a548fc3f43c/users/f965c9163ebf41a9866af7389feaf17c/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:14.567655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f96bde8-5d3d-4692-b77c-1b2e339c6bfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c24a78af0a5d46de89b42a548fc3f43c, user_id=f965c9163ebf41a9866af7389feaf17c, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:14.570347 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f96bde8-5d3d-4692-b77c-1b2e339c6bfc admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:14.583490 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1259/2497] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:14 2022] PUT /identity/v3/projects/c24a78af0a5d46de89b42a548fc3f43c/users/f965c9163ebf41a9866af7389feaf17c/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:43:14.588456 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8f0a1-b6c2-4184-917d-1b37c548ec5d None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:14.588747 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8f0a1-b6c2-4184-917d-1b37c548ec5d None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:14.589065 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8f0a1-b6c2-4184-917d-1b37c548ec5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:14.608663 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-c1e8f0a1-b6c2-4184-917d-1b37c548ec5d None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:14.609909 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c1e8f0a1-b6c2-4184-917d-1b37c548ec5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:14.622390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:14.668652 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1239/2498] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:14.678180 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96bde8-5d3d-4692-b77c-1b2e339c6bfc admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:14.679858 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-6f96bde8-5d3d-4692-b77c-1b2e339c6bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:14.738096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96bde8-5d3d-4692-b77c-1b2e339c6bfc admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:14.741799 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0def3bf-1290-4e74-840f-77e78c171a78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:14.742410 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0def3bf-1290-4e74-840f-77e78c171a78 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:14.742692 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0def3bf-1290-4e74-840f-77e78c171a78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:14.742967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0def3bf-1290-4e74-840f-77e78c171a78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:14.751915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0def3bf-1290-4e74-840f-77e78c171a78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:14.753328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0def3bf-1290-4e74-840f-77e78c171a78 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:14.762594 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1260/2499] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:15.240739 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8f0a1-b6c2-4184-917d-1b37c548ec5d None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:15.242402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c1e8f0a1-b6c2-4184-917d-1b37c548ec5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:15.254839 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8f0a1-b6c2-4184-917d-1b37c548ec5d None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:15.260195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70344868-b79c-4ddc-86e6-62235207b194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:15.260900 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70344868-b79c-4ddc-86e6-62235207b194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:15.261150 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70344868-b79c-4ddc-86e6-62235207b194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:15.261428 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70344868-b79c-4ddc-86e6-62235207b194 admin admin] PATH_INFO: `/v3/users/ab8684bfd1e742c494f257e1a48bb02f` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:15.262609 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70344868-b79c-4ddc-86e6-62235207b194 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab8684bfd1e742c494f257e1a48bb02f)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:15.265258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70344868-b79c-4ddc-86e6-62235207b194 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:15.314723 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-70344868-b79c-4ddc-86e6-62235207b194 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab8684bfd1e742c494f257e1a48bb02f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:15.322740 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-70344868-b79c-4ddc-86e6-62235207b194 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab8684bfd1e742c494f257e1a48bb02f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:15.330265 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-70344868-b79c-4ddc-86e6-62235207b194 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab8684bfd1e742c494f257e1a48bb02f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:15.342510 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-70344868-b79c-4ddc-86e6-62235207b194 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab8684bfd1e742c494f257e1a48bb02f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:15.343716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-70344868-b79c-4ddc-86e6-62235207b194 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab8684bfd1e742c494f257e1a48bb02f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:15.349920 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1240/2500] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:43:15 2022] DELETE /identity/v3/users/ab8684bfd1e742c494f257e1a48bb02f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:15.354137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b5fd6-6f60-4e2e-881d-0261423ec616 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:15.354426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b5fd6-6f60-4e2e-881d-0261423ec616 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:15.354703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b5fd6-6f60-4e2e-881d-0261423ec616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:15.368336 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:15.377636 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-824b5fd6-6f60-4e2e-881d-0261423ec616 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:15.379324 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-824b5fd6-6f60-4e2e-881d-0261423ec616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:15.451493 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1261/2501] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:15.462788 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70344868-b79c-4ddc-86e6-62235207b194 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:15.464362 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-70344868-b79c-4ddc-86e6-62235207b194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:15.534083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70344868-b79c-4ddc-86e6-62235207b194 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:15.546736 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c73e78-7cc6-4947-88d4-fd0622dfc488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:15.547576 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c73e78-7cc6-4947-88d4-fd0622dfc488 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:15.548790 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c73e78-7cc6-4947-88d4-fd0622dfc488 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:15.549218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c73e78-7cc6-4947-88d4-fd0622dfc488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:15.561509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c73e78-7cc6-4947-88d4-fd0622dfc488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:15.563203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c73e78-7cc6-4947-88d4-fd0622dfc488 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:15.573979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1241/2502] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:43:15.898541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b5fd6-6f60-4e2e-881d-0261423ec616 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:15.900161 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-824b5fd6-6f60-4e2e-881d-0261423ec616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:15.917552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b5fd6-6f60-4e2e-881d-0261423ec616 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:15.924487 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc32f8-b831-491d-9e8a-dd8017aa8cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:15.925409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc32f8-b831-491d-9e8a-dd8017aa8cfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:15.925872 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc32f8-b831-491d-9e8a-dd8017aa8cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:15.926924 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc32f8-b831-491d-9e8a-dd8017aa8cfd admin admin] PATH_INFO: `/v3/projects/10305a8deb1940bb89430348015a1841` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:15.928874 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc32f8-b831-491d-9e8a-dd8017aa8cfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=10305a8deb1940bb89430348015a1841)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:15.931556 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc32f8-b831-491d-9e8a-dd8017aa8cfd admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:16.002215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-95bc32f8-b831-491d-9e8a-dd8017aa8cfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10305a8deb1940bb89430348015a1841', 'request_id': 'req-95bc32f8-b831-491d-9e8a-dd8017aa8cfd'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:16.015182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-95bc32f8-b831-491d-9e8a-dd8017aa8cfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10305a8deb1940bb89430348015a1841', 'request_id': 'req-95bc32f8-b831-491d-9e8a-dd8017aa8cfd'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:16.027538 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-95bc32f8-b831-491d-9e8a-dd8017aa8cfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10305a8deb1940bb89430348015a1841', 'request_id': 'req-95bc32f8-b831-491d-9e8a-dd8017aa8cfd'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:16.046424 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-95bc32f8-b831-491d-9e8a-dd8017aa8cfd admin admin] The token cache is being invalidate because project 10305a8deb1940bb89430348015a1841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:43:16.046807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-95bc32f8-b831-491d-9e8a-dd8017aa8cfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:16.047544 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-95bc32f8-b831-491d-9e8a-dd8017aa8cfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:16.049364 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1262/2503] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:43:15 2022] DELETE /identity/v3/projects/10305a8deb1940bb89430348015a1841 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:16.053763 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c73e78-7cc6-4947-88d4-fd0622dfc488 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:16.068427 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c73e78-7cc6-4947-88d4-fd0622dfc488 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:16.108608 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:16.153031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba1d01-8711-4dbf-9cb9-d0fed9b65046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:16.153031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba1d01-8711-4dbf-9cb9-d0fed9b65046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:16.153031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba1d01-8711-4dbf-9cb9-d0fed9b65046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:16.153031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba1d01-8711-4dbf-9cb9-d0fed9b65046 admin admin] PATH_INFO: `/v3/users/94ef13aa52e74c5bbfc5e5b90043579f` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:16.154496 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba1d01-8711-4dbf-9cb9-d0fed9b65046 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94ef13aa52e74c5bbfc5e5b90043579f)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:16.156628 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba1d01-8711-4dbf-9cb9-d0fed9b65046 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:16.219531 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-4fba1d01-8711-4dbf-9cb9-d0fed9b65046 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94ef13aa52e74c5bbfc5e5b90043579f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:16.230455 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-4fba1d01-8711-4dbf-9cb9-d0fed9b65046 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94ef13aa52e74c5bbfc5e5b90043579f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:16.244369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-4fba1d01-8711-4dbf-9cb9-d0fed9b65046 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94ef13aa52e74c5bbfc5e5b90043579f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:16.267496 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-4fba1d01-8711-4dbf-9cb9-d0fed9b65046 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94ef13aa52e74c5bbfc5e5b90043579f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:16.267496 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-4fba1d01-8711-4dbf-9cb9-d0fed9b65046 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94ef13aa52e74c5bbfc5e5b90043579f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:16.270912 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1242/2504] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:43:16 2022] DELETE /identity/v3/users/94ef13aa52e74c5bbfc5e5b90043579f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:16.614746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc32f8-b831-491d-9e8a-dd8017aa8cfd admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:16.616253 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-95bc32f8-b831-491d-9e8a-dd8017aa8cfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:16.632160 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc32f8-b831-491d-9e8a-dd8017aa8cfd admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:16.751790 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f96eb-526a-4d02-942b-032aabf0df81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:16.753255 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f96eb-526a-4d02-942b-032aabf0df81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:16.753728 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f96eb-526a-4d02-942b-032aabf0df81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:16.754158 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f96eb-526a-4d02-942b-032aabf0df81 admin admin] PATH_INFO: `/v3/projects/95e45e7c145f474f803580fdc168fc1e` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:16.755948 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f96eb-526a-4d02-942b-032aabf0df81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95e45e7c145f474f803580fdc168fc1e)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:16.759055 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f96eb-526a-4d02-942b-032aabf0df81 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:16.803416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:16.820706 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-df3f96eb-526a-4d02-942b-032aabf0df81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95e45e7c145f474f803580fdc168fc1e', 'request_id': 'req-df3f96eb-526a-4d02-942b-032aabf0df81'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:16.831169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-df3f96eb-526a-4d02-942b-032aabf0df81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95e45e7c145f474f803580fdc168fc1e', 'request_id': 'req-df3f96eb-526a-4d02-942b-032aabf0df81'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:16.842213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-df3f96eb-526a-4d02-942b-032aabf0df81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95e45e7c145f474f803580fdc168fc1e', 'request_id': 'req-df3f96eb-526a-4d02-942b-032aabf0df81'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:16.855877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-df3f96eb-526a-4d02-942b-032aabf0df81 admin admin] The token cache is being invalidate because project 95e45e7c145f474f803580fdc168fc1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:43:16.856176 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-df3f96eb-526a-4d02-942b-032aabf0df81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:16.856872 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-df3f96eb-526a-4d02-942b-032aabf0df81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:16.858430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1263/2505] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:43:16 2022] DELETE /identity/v3/projects/95e45e7c145f474f803580fdc168fc1e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:16.863221 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86e12b-464c-4705-8df4-faac21c50ba7 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:16.863695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86e12b-464c-4705-8df4-faac21c50ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:16.863800 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86e12b-464c-4705-8df4-faac21c50ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:16.884867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-3e86e12b-464c-4705-8df4-faac21c50ba7 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:16.886137 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-3e86e12b-464c-4705-8df4-faac21c50ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:16.977112 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701c0b6-a69b-4686-b569-e8d2ed8bf70a None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:16.977515 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701c0b6-a69b-4686-b569-e8d2ed8bf70a None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:16.977913 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701c0b6-a69b-4686-b569-e8d2ed8bf70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:16.981537 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1243/2506] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:17.010239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-4701c0b6-a69b-4686-b569-e8d2ed8bf70a None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:17.014353 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-4701c0b6-a69b-4686-b569-e8d2ed8bf70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:17.032472 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1264/2507] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:17.036917 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86e12b-464c-4705-8df4-faac21c50ba7 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:17.051682 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86e12b-464c-4705-8df4-faac21c50ba7 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:17.057950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3d941-f6f7-467e-beda-a5e448fa213e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:17.059144 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3d941-f6f7-467e-beda-a5e448fa213e admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:17.059829 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3d941-f6f7-467e-beda-a5e448fa213e admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:17.060315 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3d941-f6f7-467e-beda-a5e448fa213e admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:17.061535 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3d941-f6f7-467e-beda-a5e448fa213e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:43:17.063508 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3d941-f6f7-467e-beda-a5e448fa213e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:17.065635 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3d941-f6f7-467e-beda-a5e448fa213e admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:17.088518 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1244/2508] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:43:17.093115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701c0b6-a69b-4686-b569-e8d2ed8bf70a None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:17.108144 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701c0b6-a69b-4686-b569-e8d2ed8bf70a None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:17.114882 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9cb01-f329-42d3-908c-45338ef0eace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:17.115600 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9cb01-f329-42d3-908c-45338ef0eace admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:17.115950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9cb01-f329-42d3-908c-45338ef0eace admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:17.116294 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9cb01-f329-42d3-908c-45338ef0eace admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:17.117139 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9cb01-f329-42d3-908c-45338ef0eace admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:17.119128 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9cb01-f329-42d3-908c-45338ef0eace admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:17.179630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1265/2509] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:43:17 2022] POST /identity/v3/projects => generated 375 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:17.181020 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3d941-f6f7-467e-beda-a5e448fa213e admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:17.208595 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3d941-f6f7-467e-beda-a5e448fa213e admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:17.285462 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:17.346255 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0d15a-ad0c-401f-b69a-fcf3d3a02a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:17.346894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0d15a-ad0c-401f-b69a-fcf3d3a02a2d admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:17.347181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0d15a-ad0c-401f-b69a-fcf3d3a02a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:17.347450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0d15a-ad0c-401f-b69a-fcf3d3a02a2d admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:17.348346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f0d15a-ad0c-401f-b69a-fcf3d3a02a2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:17.359905 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f0d15a-ad0c-401f-b69a-fcf3d3a02a2d admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:17.406136 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9cb01-f329-42d3-908c-45338ef0eace admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:17.413208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1245/2510] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:43:17 2022] POST /identity/v3/users => generated 288 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:17.418628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9cb01-f329-42d3-908c-45338ef0eace admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:17.425426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ea1be-b779-4d22-85cc-76ddbd56f517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:17.426321 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ea1be-b779-4d22-85cc-76ddbd56f517 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:17.426760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ea1be-b779-4d22-85cc-76ddbd56f517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:17.427365 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ea1be-b779-4d22-85cc-76ddbd56f517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:17.428275 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ea1be-b779-4d22-85cc-76ddbd56f517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:17.430134 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ea1be-b779-4d22-85cc-76ddbd56f517 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:17.449368 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0d15a-ad0c-401f-b69a-fcf3d3a02a2d admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:17.451191 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1266/2511] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:17.461667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0d15a-ad0c-401f-b69a-fcf3d3a02a2d admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:17.468148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a0872-50f6-4ad5-b7c0-7ad7b78f6f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:17.469140 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a0872-50f6-4ad5-b7c0-7ad7b78f6f92 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:17.469780 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a0872-50f6-4ad5-b7c0-7ad7b78f6f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:17.471583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a0872-50f6-4ad5-b7c0-7ad7b78f6f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:17.474292 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a0872-50f6-4ad5-b7c0-7ad7b78f6f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:17.475468 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a0872-50f6-4ad5-b7c0-7ad7b78f6f92 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:17.498027 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1246/2512] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:17.499500 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ea1be-b779-4d22-85cc-76ddbd56f517 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:17.510939 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ea1be-b779-4d22-85cc-76ddbd56f517 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:17.516604 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666a4b2-74f1-4566-95f7-3b7888e1b773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:17.517328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666a4b2-74f1-4566-95f7-3b7888e1b773 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:17.517616 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666a4b2-74f1-4566-95f7-3b7888e1b773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:17.517895 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666a4b2-74f1-4566-95f7-3b7888e1b773 admin admin] PATH_INFO: `/v3/projects/58541c50984d44dfb24a97316362f5f3/users/d47f733ff4fb47178d9ae6c5f1ca0d18/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:17.536402 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4666a4b2-74f1-4566-95f7-3b7888e1b773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58541c50984d44dfb24a97316362f5f3, user_id=d47f733ff4fb47178d9ae6c5f1ca0d18, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:17.540213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4666a4b2-74f1-4566-95f7-3b7888e1b773 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:17.578572 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1267/2513] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:17 2022] PUT /identity/v3/projects/58541c50984d44dfb24a97316362f5f3/users/d47f733ff4fb47178d9ae6c5f1ca0d18/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:17.578985 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef21ef-374f-4640-8ff8-656e203352e3 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:17.579284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef21ef-374f-4640-8ff8-656e203352e3 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:17.580210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef21ef-374f-4640-8ff8-656e203352e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:17.619128 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-84ef21ef-374f-4640-8ff8-656e203352e3 None None] MFA Rules not processed for user `d47f733ff4fb47178d9ae6c5f1ca0d18`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:17.621203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-84ef21ef-374f-4640-8ff8-656e203352e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:17.721174 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1247/2514] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:17.725234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666a4b2-74f1-4566-95f7-3b7888e1b773 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:17.834210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666a4b2-74f1-4566-95f7-3b7888e1b773 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:17.842642 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343820cd-1569-4b10-a4a1-93bc007a99e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:17.843665 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343820cd-1569-4b10-a4a1-93bc007a99e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:17.844169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343820cd-1569-4b10-a4a1-93bc007a99e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:17.845547 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343820cd-1569-4b10-a4a1-93bc007a99e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:17.846261 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343820cd-1569-4b10-a4a1-93bc007a99e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:17.848283 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343820cd-1569-4b10-a4a1-93bc007a99e7 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:17.887749 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1268/2515] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:43:17 2022] POST /identity/v3/projects => generated 375 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:17.892013 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef21ef-374f-4640-8ff8-656e203352e3 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:17.910471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef21ef-374f-4640-8ff8-656e203352e3 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:17.998318 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7975dc-4948-4bda-8c34-0671f9bceba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:17.998952 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7975dc-4948-4bda-8c34-0671f9bceba7 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:17.999233 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7975dc-4948-4bda-8c34-0671f9bceba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:17.999507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7975dc-4948-4bda-8c34-0671f9bceba7 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:18.000170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7975dc-4948-4bda-8c34-0671f9bceba7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:18.002393 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7975dc-4948-4bda-8c34-0671f9bceba7 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:18.029981 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1248/2516] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:43:17 2022] POST /identity/v3/users => generated 294 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:18.030893 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343820cd-1569-4b10-a4a1-93bc007a99e7 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:18.043586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343820cd-1569-4b10-a4a1-93bc007a99e7 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:18.051365 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8e9d0-361a-4a44-a4b9-5cb6aa793b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:18.052298 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8e9d0-361a-4a44-a4b9-5cb6aa793b26 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:18.052733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8e9d0-361a-4a44-a4b9-5cb6aa793b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:18.053338 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8e9d0-361a-4a44-a4b9-5cb6aa793b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:18.054581 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8e9d0-361a-4a44-a4b9-5cb6aa793b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:18.057156 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8e9d0-361a-4a44-a4b9-5cb6aa793b26 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:18.073967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1269/2517] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:43:18.075530 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7975dc-4948-4bda-8c34-0671f9bceba7 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:18.097544 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7975dc-4948-4bda-8c34-0671f9bceba7 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:18.112238 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab3125-8177-4ebe-83b9-75558d761330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:18.113962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab3125-8177-4ebe-83b9-75558d761330 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:18.115725 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab3125-8177-4ebe-83b9-75558d761330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:18.116194 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab3125-8177-4ebe-83b9-75558d761330 admin admin] PATH_INFO: `/v3/projects/25829dd708c24a3ba05dca19badff047/users/ea063178210c4e679eb8c011682f789f/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:18.142156 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ab3125-8177-4ebe-83b9-75558d761330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25829dd708c24a3ba05dca19badff047, user_id=ea063178210c4e679eb8c011682f789f, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:18.145664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ab3125-8177-4ebe-83b9-75558d761330 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:18.194450 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1249/2518] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:18 2022] PUT /identity/v3/projects/25829dd708c24a3ba05dca19badff047/users/ea063178210c4e679eb8c011682f789f/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:18.198449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8e9d0-361a-4a44-a4b9-5cb6aa793b26 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:18.229398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8e9d0-361a-4a44-a4b9-5cb6aa793b26 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:18.297022 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:18.339013 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34967c8-6044-4a30-8f2f-925bcf4fc3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:18.339713 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34967c8-6044-4a30-8f2f-925bcf4fc3af admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:18.340007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34967c8-6044-4a30-8f2f-925bcf4fc3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:18.340334 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34967c8-6044-4a30-8f2f-925bcf4fc3af admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:18.342105 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34967c8-6044-4a30-8f2f-925bcf4fc3af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:18.344021 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34967c8-6044-4a30-8f2f-925bcf4fc3af admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:18.364157 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab3125-8177-4ebe-83b9-75558d761330 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:18.376737 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab3125-8177-4ebe-83b9-75558d761330 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:18.377608 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1270/2519] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:18.382773 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f7c7a-1ee8-4786-9b30-e512d3b064a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:18.383732 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f7c7a-1ee8-4786-9b30-e512d3b064a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:18.384446 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f7c7a-1ee8-4786-9b30-e512d3b064a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:18.384939 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f7c7a-1ee8-4786-9b30-e512d3b064a9 admin admin] PATH_INFO: `/v3/projects/25829dd708c24a3ba05dca19badff047/users/ea063178210c4e679eb8c011682f789f/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:18.387194 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f7c7a-1ee8-4786-9b30-e512d3b064a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25829dd708c24a3ba05dca19badff047, user_id=ea063178210c4e679eb8c011682f789f, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:18.391176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f7c7a-1ee8-4786-9b30-e512d3b064a9 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:18.437433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1250/2520] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:18 2022] PUT /identity/v3/projects/25829dd708c24a3ba05dca19badff047/users/ea063178210c4e679eb8c011682f789f/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:18.438456 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385204f-45ff-45c1-ace0-679e9d6084af None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:18.438747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385204f-45ff-45c1-ace0-679e9d6084af None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:18.439204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385204f-45ff-45c1-ace0-679e9d6084af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:18.530433 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-1385204f-45ff-45c1-ace0-679e9d6084af None None] MFA Rules not processed for user `ea063178210c4e679eb8c011682f789f`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:18.532848 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-1385204f-45ff-45c1-ace0-679e9d6084af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:18.646869 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1271/2521] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:18.661879 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f7c7a-1ee8-4786-9b30-e512d3b064a9 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:18.665979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-bb9f7c7a-1ee8-4786-9b30-e512d3b064a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:18.676110 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:18.776206 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f7c7a-1ee8-4786-9b30-e512d3b064a9 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:18.783226 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58519e6d-2594-4b7d-a80c-e9fb1f554b0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:18.784593 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58519e6d-2594-4b7d-a80c-e9fb1f554b0e service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:18.785698 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58519e6d-2594-4b7d-a80c-e9fb1f554b0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:18.785983 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58519e6d-2594-4b7d-a80c-e9fb1f554b0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:18.870271 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58519e6d-2594-4b7d-a80c-e9fb1f554b0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:18.876594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58519e6d-2594-4b7d-a80c-e9fb1f554b0e service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:18.901982 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1251/2522] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:19.143802 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8c3e8-d28f-4563-804c-f47bd255dc10 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:19.144380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8c3e8-d28f-4563-804c-f47bd255dc10 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:19.144595 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8c3e8-d28f-4563-804c-f47bd255dc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:19.191216 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-e3a8c3e8-d28f-4563-804c-f47bd255dc10 None None] MFA Rules not processed for user `f965c9163ebf41a9866af7389feaf17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:19.193400 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e3a8c3e8-d28f-4563-804c-f47bd255dc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:19.300388 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1272/2523] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:19.305966 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58519e6d-2594-4b7d-a80c-e9fb1f554b0e service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:19.309454 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-58519e6d-2594-4b7d-a80c-e9fb1f554b0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:19.414608 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58519e6d-2594-4b7d-a80c-e9fb1f554b0e service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:19.419998 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66032738-1f5a-4bf7-8093-876b6473c82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:19.420628 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66032738-1f5a-4bf7-8093-876b6473c82a admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:19.420943 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66032738-1f5a-4bf7-8093-876b6473c82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:19.421220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66032738-1f5a-4bf7-8093-876b6473c82a admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:19.421884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66032738-1f5a-4bf7-8093-876b6473c82a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:19.423824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66032738-1f5a-4bf7-8093-876b6473c82a admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:19.456407 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1252/2524] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:43:19 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:19.457674 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a8c3e8-d28f-4563-804c-f47bd255dc10 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:19.470471 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a8c3e8-d28f-4563-804c-f47bd255dc10 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:19.525966 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:19.551718 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c294d-4c82-4343-ad4f-b26e871fed76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:19.552390 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c294d-4c82-4343-ad4f-b26e871fed76 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:19.552674 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c294d-4c82-4343-ad4f-b26e871fed76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:19.552985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c294d-4c82-4343-ad4f-b26e871fed76 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:19.553662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c294d-4c82-4343-ad4f-b26e871fed76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:19.555586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c294d-4c82-4343-ad4f-b26e871fed76 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:19.588600 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1273/2525] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:43:19 2022] POST /identity/v3/users => generated 296 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:19.589686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66032738-1f5a-4bf7-8093-876b6473c82a admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:19.608073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66032738-1f5a-4bf7-8093-876b6473c82a admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:19.617286 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0c2fc-6ed3-44f1-bfda-de510d561b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:19.618013 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0c2fc-6ed3-44f1-bfda-de510d561b5b admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:19.618504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0c2fc-6ed3-44f1-bfda-de510d561b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:19.618881 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0c2fc-6ed3-44f1-bfda-de510d561b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:19.619734 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0c2fc-6ed3-44f1-bfda-de510d561b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:19.621563 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0c2fc-6ed3-44f1-bfda-de510d561b5b admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:19.635019 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1253/2526] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:43:19.639377 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c294d-4c82-4343-ad4f-b26e871fed76 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:19.651629 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c294d-4c82-4343-ad4f-b26e871fed76 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:19.670652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f513d46-3aab-41d1-9fb6-5ea82167e983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:19.671354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f513d46-3aab-41d1-9fb6-5ea82167e983 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:19.671643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f513d46-3aab-41d1-9fb6-5ea82167e983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:19.671924 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f513d46-3aab-41d1-9fb6-5ea82167e983 admin admin] PATH_INFO: `/v3/projects/68d55b4deef34beba72a1f5971bed410/users/1b17a8867702402eafa70062443c46c9/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:19.702452 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f513d46-3aab-41d1-9fb6-5ea82167e983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68d55b4deef34beba72a1f5971bed410, user_id=1b17a8867702402eafa70062443c46c9, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:19.705485 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f513d46-3aab-41d1-9fb6-5ea82167e983 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:19.730793 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1274/2527] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:19 2022] PUT /identity/v3/projects/68d55b4deef34beba72a1f5971bed410/users/1b17a8867702402eafa70062443c46c9/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:19.735694 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0c2fc-6ed3-44f1-bfda-de510d561b5b admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:19.747367 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0c2fc-6ed3-44f1-bfda-de510d561b5b admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:19.787586 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:19.821426 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16523aa5-ad8b-43a0-9dd1-af157fe39af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:19.822155 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16523aa5-ad8b-43a0-9dd1-af157fe39af0 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:19.822513 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16523aa5-ad8b-43a0-9dd1-af157fe39af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:19.822874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16523aa5-ad8b-43a0-9dd1-af157fe39af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:19.823641 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16523aa5-ad8b-43a0-9dd1-af157fe39af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:19.825400 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16523aa5-ad8b-43a0-9dd1-af157fe39af0 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:19.836703 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1254/2528] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:19.840909 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f513d46-3aab-41d1-9fb6-5ea82167e983 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:19.855930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f513d46-3aab-41d1-9fb6-5ea82167e983 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:19.861689 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc11d1-bb93-4cda-8877-d6ac5d6a886e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:19.862359 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc11d1-bb93-4cda-8877-d6ac5d6a886e admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:19.862638 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc11d1-bb93-4cda-8877-d6ac5d6a886e admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:19.862912 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc11d1-bb93-4cda-8877-d6ac5d6a886e admin admin] PATH_INFO: `/v3/projects/68d55b4deef34beba72a1f5971bed410/users/1b17a8867702402eafa70062443c46c9/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:19.865472 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc11d1-bb93-4cda-8877-d6ac5d6a886e admin admin] RBAC: Authorizing `identity:create_grant(project_id=68d55b4deef34beba72a1f5971bed410, user_id=1b17a8867702402eafa70062443c46c9, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:19.868100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc11d1-bb93-4cda-8877-d6ac5d6a886e admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:19.881755 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1275/2529] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:19 2022] PUT /identity/v3/projects/68d55b4deef34beba72a1f5971bed410/users/1b17a8867702402eafa70062443c46c9/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:21.730457 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16523aa5-ad8b-43a0-9dd1-af157fe39af0 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:21.744829 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16523aa5-ad8b-43a0-9dd1-af157fe39af0 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:21.818822 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081d777-f204-4d87-b6e0-50b303074c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:21.819614 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081d777-f204-4d87-b6e0-50b303074c01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:21.820148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081d777-f204-4d87-b6e0-50b303074c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:21.820538 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081d777-f204-4d87-b6e0-50b303074c01 admin admin] PATH_INFO: `/v3/users/d47f733ff4fb47178d9ae6c5f1ca0d18` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:21.821896 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081d777-f204-4d87-b6e0-50b303074c01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d47f733ff4fb47178d9ae6c5f1ca0d18)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:21.823972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081d777-f204-4d87-b6e0-50b303074c01 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:21.879208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-7081d777-f204-4d87-b6e0-50b303074c01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd47f733ff4fb47178d9ae6c5f1ca0d18'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:21.888195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-7081d777-f204-4d87-b6e0-50b303074c01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd47f733ff4fb47178d9ae6c5f1ca0d18'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:21.895886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-7081d777-f204-4d87-b6e0-50b303074c01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd47f733ff4fb47178d9ae6c5f1ca0d18'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:21.908941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-7081d777-f204-4d87-b6e0-50b303074c01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd47f733ff4fb47178d9ae6c5f1ca0d18'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:21.909737 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-7081d777-f204-4d87-b6e0-50b303074c01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd47f733ff4fb47178d9ae6c5f1ca0d18'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:21.920447 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1255/2530] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:43:21 2022] DELETE /identity/v3/users/d47f733ff4fb47178d9ae6c5f1ca0d18 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:21.923538 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f92399-c6dd-4b3d-a20e-170fa3d1e0d7 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:21.924051 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f92399-c6dd-4b3d-a20e-170fa3d1e0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:21.924320 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f92399-c6dd-4b3d-a20e-170fa3d1e0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:21.948430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-25f92399-c6dd-4b3d-a20e-170fa3d1e0d7 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:21.949625 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-25f92399-c6dd-4b3d-a20e-170fa3d1e0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:22.020777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1276/2531] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:22.034810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081d777-f204-4d87-b6e0-50b303074c01 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:22.036287 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-7081d777-f204-4d87-b6e0-50b303074c01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:22.102099 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:22.109291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081d777-f204-4d87-b6e0-50b303074c01 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:22.114656 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3936f-25d6-4066-a6de-06bcd3b113e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:22.115544 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3936f-25d6-4066-a6de-06bcd3b113e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:22.116083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3936f-25d6-4066-a6de-06bcd3b113e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:22.116628 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3936f-25d6-4066-a6de-06bcd3b113e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:22.127274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd3936f-25d6-4066-a6de-06bcd3b113e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:22.128812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd3936f-25d6-4066-a6de-06bcd3b113e9 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:22.140187 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1256/2532] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:22.616038 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f92399-c6dd-4b3d-a20e-170fa3d1e0d7 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:22.617552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-25f92399-c6dd-4b3d-a20e-170fa3d1e0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:22.641913 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f92399-c6dd-4b3d-a20e-170fa3d1e0d7 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:22.652819 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01723bf6-b13b-41ba-9d6b-61aedd830953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:22.653583 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01723bf6-b13b-41ba-9d6b-61aedd830953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:22.653858 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01723bf6-b13b-41ba-9d6b-61aedd830953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:22.654132 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01723bf6-b13b-41ba-9d6b-61aedd830953 admin admin] PATH_INFO: `/v3/projects/58541c50984d44dfb24a97316362f5f3` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:22.656234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01723bf6-b13b-41ba-9d6b-61aedd830953 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58541c50984d44dfb24a97316362f5f3)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:22.657869 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01723bf6-b13b-41ba-9d6b-61aedd830953 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:22.793560 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-01723bf6-b13b-41ba-9d6b-61aedd830953 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58541c50984d44dfb24a97316362f5f3', 'request_id': 'req-01723bf6-b13b-41ba-9d6b-61aedd830953'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:22.804438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-01723bf6-b13b-41ba-9d6b-61aedd830953 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58541c50984d44dfb24a97316362f5f3', 'request_id': 'req-01723bf6-b13b-41ba-9d6b-61aedd830953'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:22.813020 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-01723bf6-b13b-41ba-9d6b-61aedd830953 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58541c50984d44dfb24a97316362f5f3', 'request_id': 'req-01723bf6-b13b-41ba-9d6b-61aedd830953'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:22.830069 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-01723bf6-b13b-41ba-9d6b-61aedd830953 admin admin] The token cache is being invalidate because project 58541c50984d44dfb24a97316362f5f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:43:22.830370 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-01723bf6-b13b-41ba-9d6b-61aedd830953 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:22.831054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-01723bf6-b13b-41ba-9d6b-61aedd830953 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:22.832936 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1277/2533] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:43:22 2022] DELETE /identity/v3/projects/58541c50984d44dfb24a97316362f5f3 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:22.836504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3936f-25d6-4066-a6de-06bcd3b113e9 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:22.848749 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3936f-25d6-4066-a6de-06bcd3b113e9 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:22.943788 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27403f95-3300-4290-b64f-d39121b6f9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:22.944877 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27403f95-3300-4290-b64f-d39121b6f9cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:22.945379 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27403f95-3300-4290-b64f-d39121b6f9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:22.945815 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27403f95-3300-4290-b64f-d39121b6f9cd admin admin] PATH_INFO: `/v3/users/ea063178210c4e679eb8c011682f789f` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:22.947266 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27403f95-3300-4290-b64f-d39121b6f9cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea063178210c4e679eb8c011682f789f)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:22.950033 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27403f95-3300-4290-b64f-d39121b6f9cd admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:23.016399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-27403f95-3300-4290-b64f-d39121b6f9cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea063178210c4e679eb8c011682f789f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:23.030104 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-27403f95-3300-4290-b64f-d39121b6f9cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea063178210c4e679eb8c011682f789f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:23.040300 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-27403f95-3300-4290-b64f-d39121b6f9cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea063178210c4e679eb8c011682f789f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:23.062129 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-27403f95-3300-4290-b64f-d39121b6f9cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea063178210c4e679eb8c011682f789f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:23.062937 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-27403f95-3300-4290-b64f-d39121b6f9cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea063178210c4e679eb8c011682f789f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:23.071941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1257/2534] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:43:22 2022] DELETE /identity/v3/users/ea063178210c4e679eb8c011682f789f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:43:23.837145 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01723bf6-b13b-41ba-9d6b-61aedd830953 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:23.839228 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-01723bf6-b13b-41ba-9d6b-61aedd830953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:23.875143 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01723bf6-b13b-41ba-9d6b-61aedd830953 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:23.917185 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:24.066742 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd272080-c8ae-43bf-ae22-6648d6bcb796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:24.067546 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd272080-c8ae-43bf-ae22-6648d6bcb796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:24.067913 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd272080-c8ae-43bf-ae22-6648d6bcb796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:24.068262 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd272080-c8ae-43bf-ae22-6648d6bcb796 admin admin] PATH_INFO: `/v3/projects/25829dd708c24a3ba05dca19badff047` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:24.073143 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd272080-c8ae-43bf-ae22-6648d6bcb796 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25829dd708c24a3ba05dca19badff047)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:24.074823 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd272080-c8ae-43bf-ae22-6648d6bcb796 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:24.222175 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-fd272080-c8ae-43bf-ae22-6648d6bcb796 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25829dd708c24a3ba05dca19badff047', 'request_id': 'req-fd272080-c8ae-43bf-ae22-6648d6bcb796'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:24.232465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-fd272080-c8ae-43bf-ae22-6648d6bcb796 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25829dd708c24a3ba05dca19badff047', 'request_id': 'req-fd272080-c8ae-43bf-ae22-6648d6bcb796'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:24.243006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-fd272080-c8ae-43bf-ae22-6648d6bcb796 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25829dd708c24a3ba05dca19badff047', 'request_id': 'req-fd272080-c8ae-43bf-ae22-6648d6bcb796'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:24.259495 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-fd272080-c8ae-43bf-ae22-6648d6bcb796 admin admin] The token cache is being invalidate because project 25829dd708c24a3ba05dca19badff047 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:43:24.260831 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-fd272080-c8ae-43bf-ae22-6648d6bcb796 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:24.262292 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-fd272080-c8ae-43bf-ae22-6648d6bcb796 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:24.265028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1278/2535] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:43:23 2022] DELETE /identity/v3/projects/25829dd708c24a3ba05dca19badff047 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:24.269992 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37b238-a9b4-457b-b549-1565b320638f None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:24.272873 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37b238-a9b4-457b-b549-1565b320638f None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:24.277512 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37b238-a9b4-457b-b549-1565b320638f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:24.325503 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-0b37b238-a9b4-457b-b549-1565b320638f None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:24.327458 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0b37b238-a9b4-457b-b549-1565b320638f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:24.441684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1258/2536] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:24.449315 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e3771-1aed-4d8b-98a0-ca8ebe2f5328 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:24.449673 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e3771-1aed-4d8b-98a0-ca8ebe2f5328 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:24.449946 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e3771-1aed-4d8b-98a0-ca8ebe2f5328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:24.489243 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-1b8e3771-1aed-4d8b-98a0-ca8ebe2f5328 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:24.490579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-1b8e3771-1aed-4d8b-98a0-ca8ebe2f5328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:24.527192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1279/2537] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:24.529523 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37b238-a9b4-457b-b549-1565b320638f None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:24.550233 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37b238-a9b4-457b-b549-1565b320638f None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:24.565510 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f240256-9782-4e2a-9d41-67c7c40cf274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:24.566162 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f240256-9782-4e2a-9d41-67c7c40cf274 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:24.566452 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f240256-9782-4e2a-9d41-67c7c40cf274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:24.566734 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f240256-9782-4e2a-9d41-67c7c40cf274 admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:24.567647 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f240256-9782-4e2a-9d41-67c7c40cf274 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:43:24.567759 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f240256-9782-4e2a-9d41-67c7c40cf274 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:24.569468 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f240256-9782-4e2a-9d41-67c7c40cf274 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:24.578892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:24.604591 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1259/2538] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:43:24.607459 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e3771-1aed-4d8b-98a0-ca8ebe2f5328 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:24.621786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e3771-1aed-4d8b-98a0-ca8ebe2f5328 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:24.644813 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2835205-fbf5-4b6a-9525-77d2bdf2a373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:24.645526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2835205-fbf5-4b6a-9525-77d2bdf2a373 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:24.645808 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2835205-fbf5-4b6a-9525-77d2bdf2a373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:24.646085 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2835205-fbf5-4b6a-9525-77d2bdf2a373 admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:24.646772 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2835205-fbf5-4b6a-9525-77d2bdf2a373 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:24.648719 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2835205-fbf5-4b6a-9525-77d2bdf2a373 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:24.705699 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1280/2539] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:43:24 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:24.709494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f240256-9782-4e2a-9d41-67c7c40cf274 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:24.731164 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f240256-9782-4e2a-9d41-67c7c40cf274 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:24.927498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9875477-351f-4824-9dd1-795ee3bb51ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:24.928709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9875477-351f-4824-9dd1-795ee3bb51ad admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:24.928962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9875477-351f-4824-9dd1-795ee3bb51ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:24.929247 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9875477-351f-4824-9dd1-795ee3bb51ad admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:24.929941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9875477-351f-4824-9dd1-795ee3bb51ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:24.935282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9875477-351f-4824-9dd1-795ee3bb51ad admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:25.009970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1260/2540] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:43:24 2022] POST /identity/v3/users => generated 289 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:25.014901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2835205-fbf5-4b6a-9525-77d2bdf2a373 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:25.031526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2835205-fbf5-4b6a-9525-77d2bdf2a373 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:25.054531 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c371b12-99db-4789-8f1f-52232898ae78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:25.063995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c371b12-99db-4789-8f1f-52232898ae78 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:25.065421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c371b12-99db-4789-8f1f-52232898ae78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:25.066037 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c371b12-99db-4789-8f1f-52232898ae78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:25.069025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c371b12-99db-4789-8f1f-52232898ae78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:25.071695 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c371b12-99db-4789-8f1f-52232898ae78 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:25.086873 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1281/2541] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:25.091462 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9875477-351f-4824-9dd1-795ee3bb51ad admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:25.110330 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9875477-351f-4824-9dd1-795ee3bb51ad admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:25.139969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabc424-366f-40e0-bb8c-f634bb588206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:25.141006 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabc424-366f-40e0-bb8c-f634bb588206 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:25.141006 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabc424-366f-40e0-bb8c-f634bb588206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:25.141395 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabc424-366f-40e0-bb8c-f634bb588206 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:25.142154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaabc424-366f-40e0-bb8c-f634bb588206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:25.147565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaabc424-366f-40e0-bb8c-f634bb588206 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:25.186382 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1261/2542] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:25.187766 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c371b12-99db-4789-8f1f-52232898ae78 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:25.213278 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c371b12-99db-4789-8f1f-52232898ae78 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:25.220726 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45775be-cdd9-4429-b7de-e80cfc52005c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:25.221617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45775be-cdd9-4429-b7de-e80cfc52005c admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:25.222149 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45775be-cdd9-4429-b7de-e80cfc52005c admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:25.222430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45775be-cdd9-4429-b7de-e80cfc52005c admin admin] PATH_INFO: `/v3/projects/60b34481ed584cd8ae545b21e40b8fc1/users/da95ef69d1df4860995d502dcfb28856/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:25.249398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45775be-cdd9-4429-b7de-e80cfc52005c admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b34481ed584cd8ae545b21e40b8fc1, user_id=da95ef69d1df4860995d502dcfb28856, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:25.252239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45775be-cdd9-4429-b7de-e80cfc52005c admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:25.287804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9da9b9-f06c-4ab3-b52a-c5626de046eb None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:25.288357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9da9b9-f06c-4ab3-b52a-c5626de046eb None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:25.288357 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9da9b9-f06c-4ab3-b52a-c5626de046eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:25.291988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1282/2543] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:25 2022] PUT /identity/v3/projects/60b34481ed584cd8ae545b21e40b8fc1/users/da95ef69d1df4860995d502dcfb28856/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:25.345075 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-8a9da9b9-f06c-4ab3-b52a-c5626de046eb None None] MFA Rules not processed for user `da95ef69d1df4860995d502dcfb28856`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:25.347228 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-8a9da9b9-f06c-4ab3-b52a-c5626de046eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:25.530525 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1262/2544] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:25.532978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45775be-cdd9-4429-b7de-e80cfc52005c admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:25.641570 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:25.667754 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45775be-cdd9-4429-b7de-e80cfc52005c admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:25.673979 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b48e5-9a10-4715-b023-c5aa8e4b18f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:25.674929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b48e5-9a10-4715-b023-c5aa8e4b18f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:25.675292 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b48e5-9a10-4715-b023-c5aa8e4b18f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:25.675645 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b48e5-9a10-4715-b023-c5aa8e4b18f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:25.676392 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b48e5-9a10-4715-b023-c5aa8e4b18f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:25.679416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b48e5-9a10-4715-b023-c5aa8e4b18f9 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:25.742154 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1283/2545] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:43:25 2022] POST /identity/v3/projects => generated 377 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:43:25.743391 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9da9b9-f06c-4ab3-b52a-c5626de046eb None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:25.757634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9da9b9-f06c-4ab3-b52a-c5626de046eb None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:25.891965 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa78ebd-d430-4f43-b4c5-955a569a8297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:25.892948 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa78ebd-d430-4f43-b4c5-955a569a8297 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:25.893313 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa78ebd-d430-4f43-b4c5-955a569a8297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:25.893690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa78ebd-d430-4f43-b4c5-955a569a8297 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:25.894482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa78ebd-d430-4f43-b4c5-955a569a8297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:25.896739 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa78ebd-d430-4f43-b4c5-955a569a8297 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:25.916188 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:25.941299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1263/2546] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:43:25 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:25.942802 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b48e5-9a10-4715-b023-c5aa8e4b18f9 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:25.955029 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b48e5-9a10-4715-b023-c5aa8e4b18f9 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:25.960730 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24857eb8-456e-42b0-95ef-a4d61b91e97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:25.961630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24857eb8-456e-42b0-95ef-a4d61b91e97a admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:25.962029 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24857eb8-456e-42b0-95ef-a4d61b91e97a admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:25.962276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24857eb8-456e-42b0-95ef-a4d61b91e97a admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:25.963310 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24857eb8-456e-42b0-95ef-a4d61b91e97a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:25.965142 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24857eb8-456e-42b0-95ef-a4d61b91e97a admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:25.999253 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1284/2547] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:26.003346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa78ebd-d430-4f43-b4c5-955a569a8297 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:26.025179 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa78ebd-d430-4f43-b4c5-955a569a8297 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:26.036443 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53017d7d-2579-4b14-bba1-7557aef3fe25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:26.037683 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53017d7d-2579-4b14-bba1-7557aef3fe25 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:26.037683 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53017d7d-2579-4b14-bba1-7557aef3fe25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:26.038249 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53017d7d-2579-4b14-bba1-7557aef3fe25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:26.039083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53017d7d-2579-4b14-bba1-7557aef3fe25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:26.041575 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53017d7d-2579-4b14-bba1-7557aef3fe25 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:26.077276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24857eb8-456e-42b0-95ef-a4d61b91e97a admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:26.088599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24857eb8-456e-42b0-95ef-a4d61b91e97a admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:26.088955 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1264/2548] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:26.094995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4b4a3-90cc-4479-9c97-68ef348a9bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:26.097365 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4b4a3-90cc-4479-9c97-68ef348a9bf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:26.097652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4b4a3-90cc-4479-9c97-68ef348a9bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:26.097928 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4b4a3-90cc-4479-9c97-68ef348a9bf0 admin admin] PATH_INFO: `/v3/projects/8b5ea121c01740d990ed8cb66d237193/users/05ddb1b1a2a44bea915ea68c66a43735/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:26.157546 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4b4a3-90cc-4479-9c97-68ef348a9bf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b5ea121c01740d990ed8cb66d237193, user_id=05ddb1b1a2a44bea915ea68c66a43735, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:26.160166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4b4a3-90cc-4479-9c97-68ef348a9bf0 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:26.185367 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1285/2549] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:26 2022] PUT /identity/v3/projects/8b5ea121c01740d990ed8cb66d237193/users/05ddb1b1a2a44bea915ea68c66a43735/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:43:26.196258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3695ee-7b62-4f1e-87c8-260a5c6ca1ff None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:26.196547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3695ee-7b62-4f1e-87c8-260a5c6ca1ff None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:26.196823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3695ee-7b62-4f1e-87c8-260a5c6ca1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:26.239104 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-5a3695ee-7b62-4f1e-87c8-260a5c6ca1ff None None] MFA Rules not processed for user `05ddb1b1a2a44bea915ea68c66a43735`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:26.240361 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-5a3695ee-7b62-4f1e-87c8-260a5c6ca1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:26.354462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2faf04-b869-4dbf-b666-14490fb3edad None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:26.354768 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2faf04-b869-4dbf-b666-14490fb3edad None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:26.355031 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2faf04-b869-4dbf-b666-14490fb3edad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:26.356986 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1265/2550] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:26.361369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3695ee-7b62-4f1e-87c8-260a5c6ca1ff None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:26.398046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-4c2faf04-b869-4dbf-b666-14490fb3edad None None] MFA Rules not processed for user `1b17a8867702402eafa70062443c46c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:26.399805 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-4c2faf04-b869-4dbf-b666-14490fb3edad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:26.446192 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3695ee-7b62-4f1e-87c8-260a5c6ca1ff None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:26.457008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7826a-f724-46f2-b590-6f01fb051b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:26.457418 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7826a-f724-46f2-b590-6f01fb051b3b admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:26.457701 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7826a-f724-46f2-b590-6f01fb051b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:26.457978 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7826a-f724-46f2-b590-6f01fb051b3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:26.458655 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7826a-f724-46f2-b590-6f01fb051b3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:26.463233 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7826a-f724-46f2-b590-6f01fb051b3b admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:26.504962 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1266/2551] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:43:26 2022] POST /identity/v3/projects => generated 377 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:26.509158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7826a-f724-46f2-b590-6f01fb051b3b admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:26.515490 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1286/2552] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:26.532110 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2faf04-b869-4dbf-b666-14490fb3edad None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:26.534864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-4c2faf04-b869-4dbf-b666-14490fb3edad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:26.599275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:26.605722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7826a-f724-46f2-b590-6f01fb051b3b admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:26.614950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4bdc9-7306-4cf5-91a8-0c1f89ab5e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:26.616345 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4bdc9-7306-4cf5-91a8-0c1f89ab5e13 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:26.616740 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4bdc9-7306-4cf5-91a8-0c1f89ab5e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:26.617170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4bdc9-7306-4cf5-91a8-0c1f89ab5e13 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:26.618227 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac4bdc9-7306-4cf5-91a8-0c1f89ab5e13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:26.621114 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac4bdc9-7306-4cf5-91a8-0c1f89ab5e13 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:26.649897 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1267/2553] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:43:26 2022] POST /identity/v3/users => generated 295 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:26.653492 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4bdc9-7306-4cf5-91a8-0c1f89ab5e13 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:26.659846 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2faf04-b869-4dbf-b666-14490fb3edad None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:26.668709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4bdc9-7306-4cf5-91a8-0c1f89ab5e13 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:26.675917 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac145d-118e-4900-a10e-151468548d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:26.677204 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac145d-118e-4900-a10e-151468548d2b admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:26.677204 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac145d-118e-4900-a10e-151468548d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:26.678528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac145d-118e-4900-a10e-151468548d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:26.678528 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac145d-118e-4900-a10e-151468548d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:26.680602 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac145d-118e-4900-a10e-151468548d2b admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:26.697457 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1268/2554] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:26.697618 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:26.698997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac145d-118e-4900-a10e-151468548d2b admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:26.712358 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac145d-118e-4900-a10e-151468548d2b admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:26.727011 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d400b-c90e-4f53-9fc1-0e99f19a4698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:26.727807 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d400b-c90e-4f53-9fc1-0e99f19a4698 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:26.728196 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d400b-c90e-4f53-9fc1-0e99f19a4698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:26.728577 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d400b-c90e-4f53-9fc1-0e99f19a4698 admin admin] PATH_INFO: `/v3/projects/296c0d90bbe14ecaaec67442f8c0d58d/users/d45aaa9915804f6bba0b41480bafe802/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:26.747584 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d400b-c90e-4f53-9fc1-0e99f19a4698 admin admin] RBAC: Authorizing `identity:create_grant(project_id=296c0d90bbe14ecaaec67442f8c0d58d, user_id=d45aaa9915804f6bba0b41480bafe802, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:26.751137 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d400b-c90e-4f53-9fc1-0e99f19a4698 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:26.772343 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1269/2555] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:26 2022] PUT /identity/v3/projects/296c0d90bbe14ecaaec67442f8c0d58d/users/d45aaa9915804f6bba0b41480bafe802/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:26.777541 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d400b-c90e-4f53-9fc1-0e99f19a4698 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:26.814710 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc3dc8-41d2-4257-8214-239c5007ad70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:26.815432 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc3dc8-41d2-4257-8214-239c5007ad70 service nova] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:26.818447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc3dc8-41d2-4257-8214-239c5007ad70 service nova] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:26.818447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc3dc8-41d2-4257-8214-239c5007ad70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:26.825393 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f3dc3dc8-41d2-4257-8214-239c5007ad70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:26.895767 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d400b-c90e-4f53-9fc1-0e99f19a4698 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:26.901405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42172b6d-c810-4c52-a266-4e7c5e3a138e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:26.902030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42172b6d-c810-4c52-a266-4e7c5e3a138e admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:26.902522 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42172b6d-c810-4c52-a266-4e7c5e3a138e admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:26.902804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42172b6d-c810-4c52-a266-4e7c5e3a138e admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:26.904063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42172b6d-c810-4c52-a266-4e7c5e3a138e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:26.910148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42172b6d-c810-4c52-a266-4e7c5e3a138e admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:26.921248 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc3dc8-41d2-4257-8214-239c5007ad70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:26.923478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc3dc8-41d2-4257-8214-239c5007ad70 service nova] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:26.925873 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1270/2556] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:26.926934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42172b6d-c810-4c52-a266-4e7c5e3a138e admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:26.938052 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42172b6d-c810-4c52-a266-4e7c5e3a138e admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:26.940562 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1287/2557] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:26.943961 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7cdd8-fec2-4d69-8625-3d1b728c2044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:26.944647 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7cdd8-fec2-4d69-8625-3d1b728c2044 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:26.944973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7cdd8-fec2-4d69-8625-3d1b728c2044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:26.945279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7cdd8-fec2-4d69-8625-3d1b728c2044 admin admin] PATH_INFO: `/v3/projects/296c0d90bbe14ecaaec67442f8c0d58d/users/d45aaa9915804f6bba0b41480bafe802/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:26.947928 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7cdd8-fec2-4d69-8625-3d1b728c2044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=296c0d90bbe14ecaaec67442f8c0d58d, user_id=d45aaa9915804f6bba0b41480bafe802, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:26.951303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7cdd8-fec2-4d69-8625-3d1b728c2044 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:26.965410 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1271/2558] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:26 2022] PUT /identity/v3/projects/296c0d90bbe14ecaaec67442f8c0d58d/users/d45aaa9915804f6bba0b41480bafe802/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:26.976579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3b0fa-941a-4bac-a12e-3f3716d88b35 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:26.976791 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3b0fa-941a-4bac-a12e-3f3716d88b35 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:26.977443 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3b0fa-941a-4bac-a12e-3f3716d88b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:27.014778 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-3ed3b0fa-941a-4bac-a12e-3f3716d88b35 None None] MFA Rules not processed for user `d45aaa9915804f6bba0b41480bafe802`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:27.016422 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-3ed3b0fa-941a-4bac-a12e-3f3716d88b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:27.199259 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1288/2559] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:26 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:27.210911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7cdd8-fec2-4d69-8625-3d1b728c2044 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:27.213289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-a1f7cdd8-fec2-4d69-8625-3d1b728c2044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:27.336857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7cdd8-fec2-4d69-8625-3d1b728c2044 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:27.350982 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9a1fe-abe9-4662-81d9-d6cd8782dc81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:27.351633 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9a1fe-abe9-4662-81d9-d6cd8782dc81 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:27.351923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9a1fe-abe9-4662-81d9-d6cd8782dc81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:27.352201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9a1fe-abe9-4662-81d9-d6cd8782dc81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:27.473623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:27.500994 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b9a1fe-abe9-4662-81d9-d6cd8782dc81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:27.502616 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b9a1fe-abe9-4662-81d9-d6cd8782dc81 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:27.537777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1272/2560] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:29.350503 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3b0fa-941a-4bac-a12e-3f3716d88b35 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:29.361661 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3b0fa-941a-4bac-a12e-3f3716d88b35 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:29.367124 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68eb14-2f46-4de4-951a-caffc1dae0e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:29.368000 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68eb14-2f46-4de4-951a-caffc1dae0e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:29.368443 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68eb14-2f46-4de4-951a-caffc1dae0e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:29.368858 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68eb14-2f46-4de4-951a-caffc1dae0e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:29.441296 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:29.467814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68eb14-2f46-4de4-951a-caffc1dae0e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:29.470010 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68eb14-2f46-4de4-951a-caffc1dae0e9 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:29.500253 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1289/2561] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:30.339537 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9a1fe-abe9-4662-81d9-d6cd8782dc81 service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:30.350866 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9a1fe-abe9-4662-81d9-d6cd8782dc81 service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:30.360256 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3335a2b-3e7a-478a-bcfc-aa5207af2565 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:30.361142 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3335a2b-3e7a-478a-bcfc-aa5207af2565 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:30.361582 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3335a2b-3e7a-478a-bcfc-aa5207af2565 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:30.362011 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3335a2b-3e7a-478a-bcfc-aa5207af2565 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:30.395469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3335a2b-3e7a-478a-bcfc-aa5207af2565 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:30.398245 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3335a2b-3e7a-478a-bcfc-aa5207af2565 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:30.427354 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1273/2562] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:30 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:31.795024 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68eb14-2f46-4de4-951a-caffc1dae0e9 service cinder] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:31.796564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-1d68eb14-2f46-4de4-951a-caffc1dae0e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:31.998263 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68eb14-2f46-4de4-951a-caffc1dae0e9 service cinder] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:32.003329 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acac29-238c-4d4e-825b-fa4e8ca105b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:32.004282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acac29-238c-4d4e-825b-fa4e8ca105b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:32.004840 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acac29-238c-4d4e-825b-fa4e8ca105b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:32.005459 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acac29-238c-4d4e-825b-fa4e8ca105b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:32.010000 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d8acac29-238c-4d4e-825b-fa4e8ca105b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:32.146805 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8acac29-238c-4d4e-825b-fa4e8ca105b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:32.148306 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8acac29-238c-4d4e-825b-fa4e8ca105b3 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:32.170530 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1290/2563] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:34.749463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3335a2b-3e7a-478a-bcfc-aa5207af2565 service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:34.761659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3335a2b-3e7a-478a-bcfc-aa5207af2565 service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:34.767963 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf396e2-e8a0-4e47-858d-ae0a05aba456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:34.769090 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf396e2-e8a0-4e47-858d-ae0a05aba456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:34.769477 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf396e2-e8a0-4e47-858d-ae0a05aba456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:34.769856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf396e2-e8a0-4e47-858d-ae0a05aba456 admin admin] PATH_INFO: `/v3/users/da95ef69d1df4860995d502dcfb28856` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:34.771166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf396e2-e8a0-4e47-858d-ae0a05aba456 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da95ef69d1df4860995d502dcfb28856)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:34.773298 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf396e2-e8a0-4e47-858d-ae0a05aba456 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:34.830950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-7cf396e2-e8a0-4e47-858d-ae0a05aba456 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da95ef69d1df4860995d502dcfb28856'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:34.840922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-7cf396e2-e8a0-4e47-858d-ae0a05aba456 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da95ef69d1df4860995d502dcfb28856'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:34.852595 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-7cf396e2-e8a0-4e47-858d-ae0a05aba456 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da95ef69d1df4860995d502dcfb28856'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:34.867659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-7cf396e2-e8a0-4e47-858d-ae0a05aba456 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da95ef69d1df4860995d502dcfb28856'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:34.868441 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-7cf396e2-e8a0-4e47-858d-ae0a05aba456 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da95ef69d1df4860995d502dcfb28856'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:34.877574 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1274/2564] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:43:34 2022] DELETE /identity/v3/users/da95ef69d1df4860995d502dcfb28856 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:34.879480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c91d3-f86b-40c7-9ba9-2db87d8df116 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:34.879908 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c91d3-f86b-40c7-9ba9-2db87d8df116 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:34.880163 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c91d3-f86b-40c7-9ba9-2db87d8df116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:34.906516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-c42c91d3-f86b-40c7-9ba9-2db87d8df116 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:34.907912 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-c42c91d3-f86b-40c7-9ba9-2db87d8df116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:34.968178 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1291/2565] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:34.979951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf396e2-e8a0-4e47-858d-ae0a05aba456 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:34.981401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-7cf396e2-e8a0-4e47-858d-ae0a05aba456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:35.086308 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf396e2-e8a0-4e47-858d-ae0a05aba456 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:35.092190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444a5d4-3eab-423b-a611-c08d12eb360e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:35.093454 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444a5d4-3eab-423b-a611-c08d12eb360e service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:35.093454 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444a5d4-3eab-423b-a611-c08d12eb360e service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:35.093676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444a5d4-3eab-423b-a611-c08d12eb360e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:35.116191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9444a5d4-3eab-423b-a611-c08d12eb360e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:35.118015 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9444a5d4-3eab-423b-a611-c08d12eb360e service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:35.133797 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1275/2566] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:35.208004 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c91d3-f86b-40c7-9ba9-2db87d8df116 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:35.209953 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-c42c91d3-f86b-40c7-9ba9-2db87d8df116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:35.225583 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c91d3-f86b-40c7-9ba9-2db87d8df116 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:35.229213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8686d11-2ec7-4948-adee-2cf86fa6f580 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:35.229836 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8686d11-2ec7-4948-adee-2cf86fa6f580 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:35.230116 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8686d11-2ec7-4948-adee-2cf86fa6f580 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:35.230389 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8686d11-2ec7-4948-adee-2cf86fa6f580 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:35.233540 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d8686d11-2ec7-4948-adee-2cf86fa6f580 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:35.245939 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8686d11-2ec7-4948-adee-2cf86fa6f580 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:35.248142 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8686d11-2ec7-4948-adee-2cf86fa6f580 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:35.258498 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1292/2567] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:35.803082 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444a5d4-3eab-423b-a611-c08d12eb360e service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:35.805284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9444a5d4-3eab-423b-a611-c08d12eb360e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:35.821385 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444a5d4-3eab-423b-a611-c08d12eb360e service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:35.827317 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f9151-75d2-4a26-b6b3-fc9d376ba8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:35.828479 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f9151-75d2-4a26-b6b3-fc9d376ba8f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:35.828841 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f9151-75d2-4a26-b6b3-fc9d376ba8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:35.829195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f9151-75d2-4a26-b6b3-fc9d376ba8f5 admin admin] PATH_INFO: `/v3/users/7a0bd78165c94d9e9750fe4af4f7d375` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:35.830485 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0f9151-75d2-4a26-b6b3-fc9d376ba8f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a0bd78165c94d9e9750fe4af4f7d375)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:35.832757 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0f9151-75d2-4a26-b6b3-fc9d376ba8f5 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:35.897143 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:35.899419 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0c0f9151-75d2-4a26-b6b3-fc9d376ba8f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a0bd78165c94d9e9750fe4af4f7d375'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:35.909247 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0c0f9151-75d2-4a26-b6b3-fc9d376ba8f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a0bd78165c94d9e9750fe4af4f7d375'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:35.921840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0c0f9151-75d2-4a26-b6b3-fc9d376ba8f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a0bd78165c94d9e9750fe4af4f7d375'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:35.945973 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8686d11-2ec7-4948-adee-2cf86fa6f580 service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:35.947610 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0c0f9151-75d2-4a26-b6b3-fc9d376ba8f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a0bd78165c94d9e9750fe4af4f7d375'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:35.948418 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d8686d11-2ec7-4948-adee-2cf86fa6f580 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:35.948568 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0c0f9151-75d2-4a26-b6b3-fc9d376ba8f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a0bd78165c94d9e9750fe4af4f7d375'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:35.967332 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1276/2568] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:43:35 2022] DELETE /identity/v3/users/7a0bd78165c94d9e9750fe4af4f7d375 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:35.971258 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8686d11-2ec7-4948-adee-2cf86fa6f580 service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:35.986972 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:36.047897 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41e6c0-5706-4f3a-ae44-a6e71189761d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:36.048536 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41e6c0-5706-4f3a-ae44-a6e71189761d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:36.048813 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41e6c0-5706-4f3a-ae44-a6e71189761d admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:36.049142 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41e6c0-5706-4f3a-ae44-a6e71189761d admin admin] PATH_INFO: `/v3/projects/60b34481ed584cd8ae545b21e40b8fc1` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:36.050290 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41e6c0-5706-4f3a-ae44-a6e71189761d admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b34481ed584cd8ae545b21e40b8fc1)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:36.052166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41e6c0-5706-4f3a-ae44-a6e71189761d admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:36.099423 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ad41e6c0-5706-4f3a-ae44-a6e71189761d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b34481ed584cd8ae545b21e40b8fc1', 'request_id': 'req-ad41e6c0-5706-4f3a-ae44-a6e71189761d'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:36.113797 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ad41e6c0-5706-4f3a-ae44-a6e71189761d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b34481ed584cd8ae545b21e40b8fc1', 'request_id': 'req-ad41e6c0-5706-4f3a-ae44-a6e71189761d'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:36.122076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ad41e6c0-5706-4f3a-ae44-a6e71189761d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b34481ed584cd8ae545b21e40b8fc1', 'request_id': 'req-ad41e6c0-5706-4f3a-ae44-a6e71189761d'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:36.136450 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ad41e6c0-5706-4f3a-ae44-a6e71189761d admin admin] The token cache is being invalidate because project 60b34481ed584cd8ae545b21e40b8fc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:43:36.136744 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ad41e6c0-5706-4f3a-ae44-a6e71189761d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:36.137444 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ad41e6c0-5706-4f3a-ae44-a6e71189761d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:36.139139 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1293/2569] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:43:35 2022] DELETE /identity/v3/projects/60b34481ed584cd8ae545b21e40b8fc1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:36.143216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f9151-75d2-4a26-b6b3-fc9d376ba8f5 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:36.145466 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0c0f9151-75d2-4a26-b6b3-fc9d376ba8f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:36.168535 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f9151-75d2-4a26-b6b3-fc9d376ba8f5 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:36.235546 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae9519-db70-4967-b8b1-83564e1037e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:36.236181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae9519-db70-4967-b8b1-83564e1037e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:36.236480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae9519-db70-4967-b8b1-83564e1037e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:36.236756 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae9519-db70-4967-b8b1-83564e1037e3 admin admin] PATH_INFO: `/v3/users/05ddb1b1a2a44bea915ea68c66a43735` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:36.237982 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ae9519-db70-4967-b8b1-83564e1037e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05ddb1b1a2a44bea915ea68c66a43735)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:36.239966 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ae9519-db70-4967-b8b1-83564e1037e3 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:36.299805 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-56ae9519-db70-4967-b8b1-83564e1037e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05ddb1b1a2a44bea915ea68c66a43735'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:36.307621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-56ae9519-db70-4967-b8b1-83564e1037e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05ddb1b1a2a44bea915ea68c66a43735'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:36.315552 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-56ae9519-db70-4967-b8b1-83564e1037e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05ddb1b1a2a44bea915ea68c66a43735'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:36.328580 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-56ae9519-db70-4967-b8b1-83564e1037e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05ddb1b1a2a44bea915ea68c66a43735'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:36.329292 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-56ae9519-db70-4967-b8b1-83564e1037e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05ddb1b1a2a44bea915ea68c66a43735'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:36.335695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1277/2570] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:43:36 2022] DELETE /identity/v3/users/05ddb1b1a2a44bea915ea68c66a43735 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:36.344102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41e6c0-5706-4f3a-ae44-a6e71189761d admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:36.345856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-ad41e6c0-5706-4f3a-ae44-a6e71189761d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:36.360932 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41e6c0-5706-4f3a-ae44-a6e71189761d admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:36.458082 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc9d9e-2ae5-40e4-aa26-58a0cc36d8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:36.458795 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc9d9e-2ae5-40e4-aa26-58a0cc36d8b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:36.459487 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc9d9e-2ae5-40e4-aa26-58a0cc36d8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:36.459487 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc9d9e-2ae5-40e4-aa26-58a0cc36d8b9 admin admin] PATH_INFO: `/v3/projects/89ddf8711f2f4be88304f967149c3a50` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:36.464245 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc9d9e-2ae5-40e4-aa26-58a0cc36d8b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89ddf8711f2f4be88304f967149c3a50)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:36.466302 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc9d9e-2ae5-40e4-aa26-58a0cc36d8b9 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:36.542852 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:36.546849 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-51dc9d9e-2ae5-40e4-aa26-58a0cc36d8b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89ddf8711f2f4be88304f967149c3a50', 'request_id': 'req-51dc9d9e-2ae5-40e4-aa26-58a0cc36d8b9'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:36.568415 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-51dc9d9e-2ae5-40e4-aa26-58a0cc36d8b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89ddf8711f2f4be88304f967149c3a50', 'request_id': 'req-51dc9d9e-2ae5-40e4-aa26-58a0cc36d8b9'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:36.580022 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-51dc9d9e-2ae5-40e4-aa26-58a0cc36d8b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89ddf8711f2f4be88304f967149c3a50', 'request_id': 'req-51dc9d9e-2ae5-40e4-aa26-58a0cc36d8b9'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:36.598307 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-51dc9d9e-2ae5-40e4-aa26-58a0cc36d8b9 admin admin] The token cache is being invalidate because project 89ddf8711f2f4be88304f967149c3a50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:43:36.598915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-51dc9d9e-2ae5-40e4-aa26-58a0cc36d8b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:36.599636 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-51dc9d9e-2ae5-40e4-aa26-58a0cc36d8b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:36.602075 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1294/2571] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:43:36 2022] DELETE /identity/v3/projects/89ddf8711f2f4be88304f967149c3a50 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:36.606928 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c755c22-b4c4-4efd-a0a2-c29e3e04c13e None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:36.607218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c755c22-b4c4-4efd-a0a2-c29e3e04c13e None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:36.607519 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c755c22-b4c4-4efd-a0a2-c29e3e04c13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:36.630262 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-4c755c22-b4c4-4efd-a0a2-c29e3e04c13e None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:36.631516 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-4c755c22-b4c4-4efd-a0a2-c29e3e04c13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:36.685941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:36.720618 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1278/2572] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:36.729119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c07cd7-2beb-48c7-9ceb-1d5572cb2698 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:36.729405 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c07cd7-2beb-48c7-9ceb-1d5572cb2698 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:36.729679 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c07cd7-2beb-48c7-9ceb-1d5572cb2698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:36.760693 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-c0c07cd7-2beb-48c7-9ceb-1d5572cb2698 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:36.762506 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-c0c07cd7-2beb-48c7-9ceb-1d5572cb2698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:36.784627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1295/2573] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:36.789018 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c755c22-b4c4-4efd-a0a2-c29e3e04c13e None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:36.801894 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c755c22-b4c4-4efd-a0a2-c29e3e04c13e None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:36.807041 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0eaee-50dc-482a-b913-4d7b7237463c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:36.807735 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0eaee-50dc-482a-b913-4d7b7237463c admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:36.808091 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0eaee-50dc-482a-b913-4d7b7237463c admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:36.808434 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0eaee-50dc-482a-b913-4d7b7237463c admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:36.809850 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0eaee-50dc-482a-b913-4d7b7237463c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:43:36.810223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0eaee-50dc-482a-b913-4d7b7237463c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:36.811897 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0eaee-50dc-482a-b913-4d7b7237463c admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:36.824775 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1279/2574] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:43:36.829659 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c07cd7-2beb-48c7-9ceb-1d5572cb2698 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:36.840173 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c07cd7-2beb-48c7-9ceb-1d5572cb2698 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:36.844932 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3f2a5-1900-47cb-bb94-57f073c63c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:36.845805 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3f2a5-1900-47cb-bb94-57f073c63c16 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:36.846087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3f2a5-1900-47cb-bb94-57f073c63c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:36.846484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3f2a5-1900-47cb-bb94-57f073c63c16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:36.847155 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3f2a5-1900-47cb-bb94-57f073c63c16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:36.849106 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3f2a5-1900-47cb-bb94-57f073c63c16 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:36.896032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1296/2575] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:43:36 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:36.900410 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0eaee-50dc-482a-b913-4d7b7237463c admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:36.911821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0eaee-50dc-482a-b913-4d7b7237463c admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:36.975525 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef4a52-1cd6-454c-adb7-13dfb5952e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:36.976400 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef4a52-1cd6-454c-adb7-13dfb5952e4d admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:36.976557 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef4a52-1cd6-454c-adb7-13dfb5952e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:36.976816 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef4a52-1cd6-454c-adb7-13dfb5952e4d admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:36.978175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef4a52-1cd6-454c-adb7-13dfb5952e4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:36.980123 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3f2a5-1900-47cb-bb94-57f073c63c16 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:36.980611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef4a52-1cd6-454c-adb7-13dfb5952e4d admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:36.983169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-62d3f2a5-1900-47cb-bb94-57f073c63c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:36.998151 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3f2a5-1900-47cb-bb94-57f073c63c16 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:37.003428 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe03ae-8006-4962-97d5-e447374e8cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:37.004082 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe03ae-8006-4962-97d5-e447374e8cb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:37.004356 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe03ae-8006-4962-97d5-e447374e8cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:37.004806 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe03ae-8006-4962-97d5-e447374e8cb2 admin admin] PATH_INFO: `/v3/projects/8b5ea121c01740d990ed8cb66d237193` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:37.006561 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe03ae-8006-4962-97d5-e447374e8cb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b5ea121c01740d990ed8cb66d237193)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:37.008748 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe03ae-8006-4962-97d5-e447374e8cb2 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:37.013533 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1280/2576] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:43:36 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:43:37.014650 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef4a52-1cd6-454c-adb7-13dfb5952e4d admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:37.027628 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef4a52-1cd6-454c-adb7-13dfb5952e4d admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:37.034014 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bc8f4-4ed7-4cd2-8140-5c2a0fa4921a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:37.034663 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bc8f4-4ed7-4cd2-8140-5c2a0fa4921a admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:37.035074 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bc8f4-4ed7-4cd2-8140-5c2a0fa4921a admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:37.035360 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bc8f4-4ed7-4cd2-8140-5c2a0fa4921a admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:37.036073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9bc8f4-4ed7-4cd2-8140-5c2a0fa4921a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:37.037944 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9bc8f4-4ed7-4cd2-8140-5c2a0fa4921a admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:37.052724 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1281/2577] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:37.056532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bc8f4-4ed7-4cd2-8140-5c2a0fa4921a admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:37.075194 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bc8f4-4ed7-4cd2-8140-5c2a0fa4921a admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:37.084354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-3abe03ae-8006-4962-97d5-e447374e8cb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b5ea121c01740d990ed8cb66d237193', 'request_id': 'req-3abe03ae-8006-4962-97d5-e447374e8cb2'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:37.093338 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-3abe03ae-8006-4962-97d5-e447374e8cb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b5ea121c01740d990ed8cb66d237193', 'request_id': 'req-3abe03ae-8006-4962-97d5-e447374e8cb2'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:37.100518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-3abe03ae-8006-4962-97d5-e447374e8cb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b5ea121c01740d990ed8cb66d237193', 'request_id': 'req-3abe03ae-8006-4962-97d5-e447374e8cb2'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:37.117029 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-3abe03ae-8006-4962-97d5-e447374e8cb2 admin admin] The token cache is being invalidate because project 8b5ea121c01740d990ed8cb66d237193 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:43:37.117318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-3abe03ae-8006-4962-97d5-e447374e8cb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:37.117975 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-3abe03ae-8006-4962-97d5-e447374e8cb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:37.120251 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1297/2578] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:43:36 2022] DELETE /identity/v3/projects/8b5ea121c01740d990ed8cb66d237193 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:37.124500 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe03ae-8006-4962-97d5-e447374e8cb2 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:37.125965 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-3abe03ae-8006-4962-97d5-e447374e8cb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:37.147409 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a30fea-95db-453c-8eb2-40cdd53eb25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:37.148047 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a30fea-95db-453c-8eb2-40cdd53eb25d admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:37.148329 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a30fea-95db-453c-8eb2-40cdd53eb25d admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:37.148652 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a30fea-95db-453c-8eb2-40cdd53eb25d admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:37.149507 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a30fea-95db-453c-8eb2-40cdd53eb25d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:37.151447 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a30fea-95db-453c-8eb2-40cdd53eb25d admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:37.173353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1282/2579] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:43:37.177606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a30fea-95db-453c-8eb2-40cdd53eb25d admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:37.180261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b7a30fea-95db-453c-8eb2-40cdd53eb25d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:37.198002 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a30fea-95db-453c-8eb2-40cdd53eb25d admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:37.205372 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b05c83-713b-48a9-a0ff-c035ec925e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:37.206166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b05c83-713b-48a9-a0ff-c035ec925e43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:37.206612 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b05c83-713b-48a9-a0ff-c035ec925e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:37.206969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b05c83-713b-48a9-a0ff-c035ec925e43 admin admin] PATH_INFO: `/v3/projects/c93a0e8d8e004bd4bacc67363ac5a9bc/users/40e28984712b4311bf769a93b4672536/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:37.219430 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe03ae-8006-4962-97d5-e447374e8cb2 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:37.228115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3fbef6-7743-497d-8ff0-071ac3e607ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:37.228471 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b05c83-713b-48a9-a0ff-c035ec925e43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c93a0e8d8e004bd4bacc67363ac5a9bc, user_id=40e28984712b4311bf769a93b4672536, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:37.228774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3fbef6-7743-497d-8ff0-071ac3e607ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:37.229083 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3fbef6-7743-497d-8ff0-071ac3e607ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:37.229354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3fbef6-7743-497d-8ff0-071ac3e607ce admin admin] PATH_INFO: `/v3/users/d45aaa9915804f6bba0b41480bafe802` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:37.230690 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3fbef6-7743-497d-8ff0-071ac3e607ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=d45aaa9915804f6bba0b41480bafe802)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:37.231809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b05c83-713b-48a9-a0ff-c035ec925e43 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:37.233313 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3fbef6-7743-497d-8ff0-071ac3e607ce admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:37.253277 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1283/2580] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:37 2022] PUT /identity/v3/projects/c93a0e8d8e004bd4bacc67363ac5a9bc/users/40e28984712b4311bf769a93b4672536/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:43:37.263727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b74d2c-662c-4e68-a034-de6d3fee35ae None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:37.264208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b74d2c-662c-4e68-a034-de6d3fee35ae None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:37.264411 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b74d2c-662c-4e68-a034-de6d3fee35ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:37.280774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=15 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:37.300197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-0e3fbef6-7743-497d-8ff0-071ac3e607ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd45aaa9915804f6bba0b41480bafe802'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:37.311502 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-33b74d2c-662c-4e68-a034-de6d3fee35ae None None] MFA Rules not processed for user `40e28984712b4311bf769a93b4672536`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:37.313031 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-33b74d2c-662c-4e68-a034-de6d3fee35ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:37.314404 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-0e3fbef6-7743-497d-8ff0-071ac3e607ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd45aaa9915804f6bba0b41480bafe802'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:37.322977 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:37.333950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-0e3fbef6-7743-497d-8ff0-071ac3e607ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd45aaa9915804f6bba0b41480bafe802'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:37.343231 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-0e3fbef6-7743-497d-8ff0-071ac3e607ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd45aaa9915804f6bba0b41480bafe802'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:37.344485 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-0e3fbef6-7743-497d-8ff0-071ac3e607ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd45aaa9915804f6bba0b41480bafe802'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:37.357131 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1298/2581] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:43:37 2022] DELETE /identity/v3/users/d45aaa9915804f6bba0b41480bafe802 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:43:37.409796 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1284/2582] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:37.421535 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3fbef6-7743-497d-8ff0-071ac3e607ce admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:37.423253 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-0e3fbef6-7743-497d-8ff0-071ac3e607ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:37.547103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3fbef6-7743-497d-8ff0-071ac3e607ce admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:37.553552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b50e9-6741-49b5-9bc6-560f0d63faf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:37.554371 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b50e9-6741-49b5-9bc6-560f0d63faf1 service nova] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:37.554809 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b50e9-6741-49b5-9bc6-560f0d63faf1 service nova] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:37.555383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b50e9-6741-49b5-9bc6-560f0d63faf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:37.683830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b50e9-6741-49b5-9bc6-560f0d63faf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:37.686263 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b50e9-6741-49b5-9bc6-560f0d63faf1 service nova] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:37.698715 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1299/2583] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:37.827778 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b74d2c-662c-4e68-a034-de6d3fee35ae None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:37.829508 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-33b74d2c-662c-4e68-a034-de6d3fee35ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:37.942262 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b74d2c-662c-4e68-a034-de6d3fee35ae None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:38.055327 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa7ce2-5f31-4ec3-8c79-d7740f322797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:38.055995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa7ce2-5f31-4ec3-8c79-d7740f322797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:38.056278 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa7ce2-5f31-4ec3-8c79-d7740f322797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:38.056551 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa7ce2-5f31-4ec3-8c79-d7740f322797 admin admin] PATH_INFO: `/v3/projects/296c0d90bbe14ecaaec67442f8c0d58d` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:38.058422 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa7ce2-5f31-4ec3-8c79-d7740f322797 admin admin] RBAC: Authorizing `identity:delete_project(project_id=296c0d90bbe14ecaaec67442f8c0d58d)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:38.062574 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa7ce2-5f31-4ec3-8c79-d7740f322797 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:38.159241 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-69aa7ce2-5f31-4ec3-8c79-d7740f322797 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '296c0d90bbe14ecaaec67442f8c0d58d', 'request_id': 'req-69aa7ce2-5f31-4ec3-8c79-d7740f322797'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:38.174433 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-69aa7ce2-5f31-4ec3-8c79-d7740f322797 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '296c0d90bbe14ecaaec67442f8c0d58d', 'request_id': 'req-69aa7ce2-5f31-4ec3-8c79-d7740f322797'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:38.176554 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:38.188932 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-69aa7ce2-5f31-4ec3-8c79-d7740f322797 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '296c0d90bbe14ecaaec67442f8c0d58d', 'request_id': 'req-69aa7ce2-5f31-4ec3-8c79-d7740f322797'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:38.210714 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-69aa7ce2-5f31-4ec3-8c79-d7740f322797 admin admin] The token cache is being invalidate because project 296c0d90bbe14ecaaec67442f8c0d58d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:43:38.210981 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-69aa7ce2-5f31-4ec3-8c79-d7740f322797 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:38.211718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-69aa7ce2-5f31-4ec3-8c79-d7740f322797 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:38.218215 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1285/2584] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:43:37 2022] DELETE /identity/v3/projects/296c0d90bbe14ecaaec67442f8c0d58d => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:38.224946 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705c2f9-9ec5-4ba0-873f-418434e66589 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:38.225481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705c2f9-9ec5-4ba0-873f-418434e66589 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:38.225704 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705c2f9-9ec5-4ba0-873f-418434e66589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:38.257978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-9705c2f9-9ec5-4ba0-873f-418434e66589 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:38.259727 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-9705c2f9-9ec5-4ba0-873f-418434e66589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:38.332767 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:38.360914 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1300/2585] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:38.426687 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d2113-d23d-4f26-9efb-a36a578a7fa1 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:38.427087 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d2113-d23d-4f26-9efb-a36a578a7fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:38.427447 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d2113-d23d-4f26-9efb-a36a578a7fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:38.449645 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-881d2113-d23d-4f26-9efb-a36a578a7fa1 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:38.451903 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-881d2113-d23d-4f26-9efb-a36a578a7fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:38.463458 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1286/2586] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:43:38.468195 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705c2f9-9ec5-4ba0-873f-418434e66589 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:38.477986 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705c2f9-9ec5-4ba0-873f-418434e66589 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:38.482480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd0948f-d714-4905-a094-7e973e3aba4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:38.483093 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0948f-d714-4905-a094-7e973e3aba4e admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:38.483367 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0948f-d714-4905-a094-7e973e3aba4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:38.483633 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0948f-d714-4905-a094-7e973e3aba4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:38.484318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd0948f-d714-4905-a094-7e973e3aba4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:43:38.484619 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd0948f-d714-4905-a094-7e973e3aba4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:38.486233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd0948f-d714-4905-a094-7e973e3aba4e admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:38.501955 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1301/2587] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:43:38.503469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d2113-d23d-4f26-9efb-a36a578a7fa1 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:38.517800 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d2113-d23d-4f26-9efb-a36a578a7fa1 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:38.522895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9b9fe-95ca-4f8f-bceb-c569784253dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:38.523712 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9b9fe-95ca-4f8f-bceb-c569784253dc admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:38.523993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9b9fe-95ca-4f8f-bceb-c569784253dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:38.524268 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9b9fe-95ca-4f8f-bceb-c569784253dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:38.524966 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9b9fe-95ca-4f8f-bceb-c569784253dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:38.527085 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9b9fe-95ca-4f8f-bceb-c569784253dc admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:38.572977 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1287/2588] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:43:38 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:38.576796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd0948f-d714-4905-a094-7e973e3aba4e admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:38.588417 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd0948f-d714-4905-a094-7e973e3aba4e admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:38.681663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7bc63-4a99-48ab-97b5-315f22e234a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:38.683359 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7bc63-4a99-48ab-97b5-315f22e234a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:38.683993 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7bc63-4a99-48ab-97b5-315f22e234a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:38.685368 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7bc63-4a99-48ab-97b5-315f22e234a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:38.686516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d7bc63-4a99-48ab-97b5-315f22e234a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:38.688740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d7bc63-4a99-48ab-97b5-315f22e234a4 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:38.721257 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1302/2589] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:43:38 2022] POST /identity/v3/users => generated 287 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:38.722616 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9b9fe-95ca-4f8f-bceb-c569784253dc admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:38.740242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9b9fe-95ca-4f8f-bceb-c569784253dc admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:38.746222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1577f-5e54-4227-8dc1-4f1b9aeda0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:38.747751 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1577f-5e54-4227-8dc1-4f1b9aeda0ea admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:38.748063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1577f-5e54-4227-8dc1-4f1b9aeda0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:38.748451 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1577f-5e54-4227-8dc1-4f1b9aeda0ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:38.749494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1577f-5e54-4227-8dc1-4f1b9aeda0ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:38.751222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1577f-5e54-4227-8dc1-4f1b9aeda0ea admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:38.767756 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1288/2590] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:38.768913 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7bc63-4a99-48ab-97b5-315f22e234a4 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:38.779462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7bc63-4a99-48ab-97b5-315f22e234a4 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:38.784930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ae654-9f24-4298-a398-923b3b17bdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:38.785983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ae654-9f24-4298-a398-923b3b17bdf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:38.786107 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ae654-9f24-4298-a398-923b3b17bdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:38.786377 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ae654-9f24-4298-a398-923b3b17bdf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:38.787157 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ae654-9f24-4298-a398-923b3b17bdf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:38.789582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ae654-9f24-4298-a398-923b3b17bdf2 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:38.802728 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1303/2591] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:38.806819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1577f-5e54-4227-8dc1-4f1b9aeda0ea admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:38.817763 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1577f-5e54-4227-8dc1-4f1b9aeda0ea admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:38.823764 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43eba3b-1160-455e-8347-b9f673478eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:38.824631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43eba3b-1160-455e-8347-b9f673478eb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:38.825221 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43eba3b-1160-455e-8347-b9f673478eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:38.825576 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43eba3b-1160-455e-8347-b9f673478eb6 admin admin] PATH_INFO: `/v3/projects/6ad80139b6a04b62be6629aa767024f3/users/189d57780b254f90ae9120f4c31d987a/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:38.860299 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43eba3b-1160-455e-8347-b9f673478eb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ad80139b6a04b62be6629aa767024f3, user_id=189d57780b254f90ae9120f4c31d987a, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:38.864081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43eba3b-1160-455e-8347-b9f673478eb6 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:38.883456 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1289/2592] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:38 2022] PUT /identity/v3/projects/6ad80139b6a04b62be6629aa767024f3/users/189d57780b254f90ae9120f4c31d987a/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:38.893702 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f5008-4a6d-421d-9187-19767145cd4b None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:38.894024 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f5008-4a6d-421d-9187-19767145cd4b None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:38.894253 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f5008-4a6d-421d-9187-19767145cd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:38.935117 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-830f5008-4a6d-421d-9187-19767145cd4b None None] MFA Rules not processed for user `189d57780b254f90ae9120f4c31d987a`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:38.936641 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-830f5008-4a6d-421d-9187-19767145cd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:39.030917 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1304/2593] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:39.040791 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43eba3b-1160-455e-8347-b9f673478eb6 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:39.042716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b43eba3b-1160-455e-8347-b9f673478eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:39.121890 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43eba3b-1160-455e-8347-b9f673478eb6 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:39.125912 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68341886-79ce-4fba-ad36-72deb6bc9552 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:39.126743 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68341886-79ce-4fba-ad36-72deb6bc9552 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:39.126867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68341886-79ce-4fba-ad36-72deb6bc9552 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:39.127098 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68341886-79ce-4fba-ad36-72deb6bc9552 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:39.155842 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68341886-79ce-4fba-ad36-72deb6bc9552 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:39.157350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68341886-79ce-4fba-ad36-72deb6bc9552 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:39.171270 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1290/2594] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:39.525093 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f5008-4a6d-421d-9187-19767145cd4b None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:39.526562 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-830f5008-4a6d-421d-9187-19767145cd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:39.615352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:39.641063 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f5008-4a6d-421d-9187-19767145cd4b None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:39.645753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5051f-00b4-44cc-ba25-b6945698651e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:39.646372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5051f-00b4-44cc-ba25-b6945698651e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:39.646651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5051f-00b4-44cc-ba25-b6945698651e admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:39.646925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5051f-00b4-44cc-ba25-b6945698651e admin admin] PATH_INFO: `/v3/users/22bb7775bae64374b11462a53853e54f` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:39.648093 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5051f-00b4-44cc-ba25-b6945698651e admin admin] RBAC: Authorizing `identity:delete_user(user_id=22bb7775bae64374b11462a53853e54f)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:39.650116 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5051f-00b4-44cc-ba25-b6945698651e admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:39.709705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-e9c5051f-00b4-44cc-ba25-b6945698651e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22bb7775bae64374b11462a53853e54f'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:39.718297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-e9c5051f-00b4-44cc-ba25-b6945698651e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22bb7775bae64374b11462a53853e54f'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:39.749182 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-e9c5051f-00b4-44cc-ba25-b6945698651e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22bb7775bae64374b11462a53853e54f'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:39.757668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-e9c5051f-00b4-44cc-ba25-b6945698651e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22bb7775bae64374b11462a53853e54f'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:39.758455 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-e9c5051f-00b4-44cc-ba25-b6945698651e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22bb7775bae64374b11462a53853e54f'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:39.769490 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1305/2595] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:43:39 2022] DELETE /identity/v3/users/22bb7775bae64374b11462a53853e54f => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:39.954877 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68341886-79ce-4fba-ad36-72deb6bc9552 service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:39.966403 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68341886-79ce-4fba-ad36-72deb6bc9552 service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:40.026007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8f0a1-fd24-47c9-bb4a-970d51cc3bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:40.026727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8f0a1-fd24-47c9-bb4a-970d51cc3bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:40.027077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8f0a1-fd24-47c9-bb4a-970d51cc3bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:40.027355 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8f0a1-fd24-47c9-bb4a-970d51cc3bce admin admin] PATH_INFO: `/v3/projects/8d570044769a44d9b18c905284cd9947` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:40.029221 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8f0a1-fd24-47c9-bb4a-970d51cc3bce admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d570044769a44d9b18c905284cd9947)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:40.031113 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8f0a1-fd24-47c9-bb4a-970d51cc3bce admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:40.061624 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:40.078891 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-51d8f0a1-fd24-47c9-bb4a-970d51cc3bce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d570044769a44d9b18c905284cd9947', 'request_id': 'req-51d8f0a1-fd24-47c9-bb4a-970d51cc3bce'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:40.084253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-51d8f0a1-fd24-47c9-bb4a-970d51cc3bce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d570044769a44d9b18c905284cd9947', 'request_id': 'req-51d8f0a1-fd24-47c9-bb4a-970d51cc3bce'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:40.092037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-51d8f0a1-fd24-47c9-bb4a-970d51cc3bce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d570044769a44d9b18c905284cd9947', 'request_id': 'req-51d8f0a1-fd24-47c9-bb4a-970d51cc3bce'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:40.103272 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-51d8f0a1-fd24-47c9-bb4a-970d51cc3bce admin admin] The token cache is being invalidate because project 8d570044769a44d9b18c905284cd9947 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:43:40.103571 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-51d8f0a1-fd24-47c9-bb4a-970d51cc3bce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:40.104175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-51d8f0a1-fd24-47c9-bb4a-970d51cc3bce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:40.105694 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1291/2596] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:43:39 2022] DELETE /identity/v3/projects/8d570044769a44d9b18c905284cd9947 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:40.110502 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52091480-cdaf-4ce5-befe-8fa582110732 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:40.110841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52091480-cdaf-4ce5-befe-8fa582110732 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:40.111064 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52091480-cdaf-4ce5-befe-8fa582110732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:40.130623 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-52091480-cdaf-4ce5-befe-8fa582110732 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:40.131827 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-52091480-cdaf-4ce5-befe-8fa582110732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:40.198845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1306/2597] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:40.209187 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf194bb-611a-40f0-b927-7d99535887e6 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:40.209508 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf194bb-611a-40f0-b927-7d99535887e6 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:40.209793 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf194bb-611a-40f0-b927-7d99535887e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:40.230163 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-6cf194bb-611a-40f0-b927-7d99535887e6 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:40.231406 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6cf194bb-611a-40f0-b927-7d99535887e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:40.239718 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1292/2598] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:40.243841 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52091480-cdaf-4ce5-befe-8fa582110732 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:40.253567 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52091480-cdaf-4ce5-befe-8fa582110732 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:40.258066 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d92991-e588-4bc8-8788-4479a53358f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:40.258695 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d92991-e588-4bc8-8788-4479a53358f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:40.258973 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d92991-e588-4bc8-8788-4479a53358f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:40.259248 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d92991-e588-4bc8-8788-4479a53358f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:40.259930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d92991-e588-4bc8-8788-4479a53358f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:43:40.260231 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d92991-e588-4bc8-8788-4479a53358f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:40.261859 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d92991-e588-4bc8-8788-4479a53358f9 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:40.273715 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1307/2599] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:43:40.277809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf194bb-611a-40f0-b927-7d99535887e6 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:40.287942 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf194bb-611a-40f0-b927-7d99535887e6 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:40.292583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbb2b1-57cc-45b4-ac8b-2194a256d284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:40.293231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbb2b1-57cc-45b4-ac8b-2194a256d284 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:40.293526 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbb2b1-57cc-45b4-ac8b-2194a256d284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:40.293804 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbb2b1-57cc-45b4-ac8b-2194a256d284 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:40.294467 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adbb2b1-57cc-45b4-ac8b-2194a256d284 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:40.296376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adbb2b1-57cc-45b4-ac8b-2194a256d284 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:40.325916 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1293/2600] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:43:40 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:43:40.330009 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d92991-e588-4bc8-8788-4479a53358f9 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:40.340176 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d92991-e588-4bc8-8788-4479a53358f9 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:40.384226 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:40.397542 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df644b4c-f598-4452-9b4e-87bea222f58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:40.398156 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df644b4c-f598-4452-9b4e-87bea222f58a admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:40.398438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df644b4c-f598-4452-9b4e-87bea222f58a admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:40.398712 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df644b4c-f598-4452-9b4e-87bea222f58a admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:40.399362 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df644b4c-f598-4452-9b4e-87bea222f58a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:40.401306 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df644b4c-f598-4452-9b4e-87bea222f58a admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:40.421440 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1308/2601] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:43:40 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:40.425659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbb2b1-57cc-45b4-ac8b-2194a256d284 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:40.436677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbb2b1-57cc-45b4-ac8b-2194a256d284 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:40.441270 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee354f2a-6226-4b35-a8ef-41bc121cb933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:40.441866 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee354f2a-6226-4b35-a8ef-41bc121cb933 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:40.442143 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee354f2a-6226-4b35-a8ef-41bc121cb933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:40.442414 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee354f2a-6226-4b35-a8ef-41bc121cb933 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:40.443092 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee354f2a-6226-4b35-a8ef-41bc121cb933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:40.444668 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee354f2a-6226-4b35-a8ef-41bc121cb933 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:40.455005 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1294/2602] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:40.458937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df644b4c-f598-4452-9b4e-87bea222f58a admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:40.470731 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df644b4c-f598-4452-9b4e-87bea222f58a admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:40.473996 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252df728-f64d-42df-9fa2-ccb7617d8512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:40.474825 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252df728-f64d-42df-9fa2-ccb7617d8512 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:40.475103 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252df728-f64d-42df-9fa2-ccb7617d8512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:40.475498 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252df728-f64d-42df-9fa2-ccb7617d8512 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:40.476189 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252df728-f64d-42df-9fa2-ccb7617d8512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:40.477806 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252df728-f64d-42df-9fa2-ccb7617d8512 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:40.488364 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1309/2603] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:43:40.492457 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee354f2a-6226-4b35-a8ef-41bc121cb933 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:40.502807 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee354f2a-6226-4b35-a8ef-41bc121cb933 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:40.507355 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59226c76-d8a9-4749-ad3e-b2a8c764cbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:40.507998 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59226c76-d8a9-4749-ad3e-b2a8c764cbd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:40.508274 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59226c76-d8a9-4749-ad3e-b2a8c764cbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:40.508543 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59226c76-d8a9-4749-ad3e-b2a8c764cbd7 admin admin] PATH_INFO: `/v3/projects/732e82dff9c84034be19f702de536d69/users/47b4280ed1754f3d988e65a631d43d24/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:40.523654 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59226c76-d8a9-4749-ad3e-b2a8c764cbd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=732e82dff9c84034be19f702de536d69, user_id=47b4280ed1754f3d988e65a631d43d24, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:40.526493 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59226c76-d8a9-4749-ad3e-b2a8c764cbd7 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:40.539099 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1295/2604] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:40 2022] PUT /identity/v3/projects/732e82dff9c84034be19f702de536d69/users/47b4280ed1754f3d988e65a631d43d24/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:40.549429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38fb52-4744-4e65-8bc9-8d56c7126e21 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:40.549917 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38fb52-4744-4e65-8bc9-8d56c7126e21 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:40.550025 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38fb52-4744-4e65-8bc9-8d56c7126e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:40.578151 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-fe38fb52-4744-4e65-8bc9-8d56c7126e21 None None] MFA Rules not processed for user `47b4280ed1754f3d988e65a631d43d24`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:40.579294 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-fe38fb52-4744-4e65-8bc9-8d56c7126e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:40.625719 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236ec73-5cac-4702-8b90-097b31dd353d None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:40.626015 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236ec73-5cac-4702-8b90-097b31dd353d None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:40.626298 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236ec73-5cac-4702-8b90-097b31dd353d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:40.648662 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-b236ec73-5cac-4702-8b90-097b31dd353d None None] MFA Rules not processed for user `189d57780b254f90ae9120f4c31d987a`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:40.649756 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1310/2605] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:40 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:40.650040 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b236ec73-5cac-4702-8b90-097b31dd353d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:40.654423 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38fb52-4744-4e65-8bc9-8d56c7126e21 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:40.714457 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1296/2606] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:43:40.717023 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38fb52-4744-4e65-8bc9-8d56c7126e21 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:40.724996 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5991c-ccb1-4030-a7ac-38fb2d037ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:40.725631 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5991c-ccb1-4030-a7ac-38fb2d037ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:40.727462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5991c-ccb1-4030-a7ac-38fb2d037ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:40.727462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5991c-ccb1-4030-a7ac-38fb2d037ee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:40.727462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5991c-ccb1-4030-a7ac-38fb2d037ee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:40.728880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5991c-ccb1-4030-a7ac-38fb2d037ee3 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:40.760461 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1311/2607] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:43:40 2022] POST /identity/v3/projects => generated 369 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:40.764979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236ec73-5cac-4702-8b90-097b31dd353d None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:40.777698 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236ec73-5cac-4702-8b90-097b31dd353d None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:40.841492 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:40.861558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a561a1-ed79-4ca3-922e-8fd0cfa02bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:40.862185 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a561a1-ed79-4ca3-922e-8fd0cfa02bdb admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:40.862488 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a561a1-ed79-4ca3-922e-8fd0cfa02bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:40.862762 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a561a1-ed79-4ca3-922e-8fd0cfa02bdb admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:40.863425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a561a1-ed79-4ca3-922e-8fd0cfa02bdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:40.865441 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a561a1-ed79-4ca3-922e-8fd0cfa02bdb admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:40.887272 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1297/2608] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:43:40 2022] POST /identity/v3/users => generated 291 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:40.891621 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5991c-ccb1-4030-a7ac-38fb2d037ee3 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:40.901820 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5991c-ccb1-4030-a7ac-38fb2d037ee3 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:40.907052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f5c24-b371-4446-a27e-56178814bdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:40.907660 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f5c24-b371-4446-a27e-56178814bdd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:40.907937 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f5c24-b371-4446-a27e-56178814bdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:40.908206 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f5c24-b371-4446-a27e-56178814bdd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:40.908908 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f5c24-b371-4446-a27e-56178814bdd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:40.910497 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f5c24-b371-4446-a27e-56178814bdd3 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:40.920412 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1312/2609] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:40.924460 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a561a1-ed79-4ca3-922e-8fd0cfa02bdb admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:40.934760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a561a1-ed79-4ca3-922e-8fd0cfa02bdb admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:40.939632 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d72474-1c08-44b7-8259-7b41a9f14866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:40.940368 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d72474-1c08-44b7-8259-7b41a9f14866 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:40.940686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d72474-1c08-44b7-8259-7b41a9f14866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:40.940994 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d72474-1c08-44b7-8259-7b41a9f14866 admin admin] PATH_INFO: `/v3/projects/fcf48935946543e8a55c5df282cefe76/users/5a9add3165a545019bfb94874a8812e4/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:40.956674 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d72474-1c08-44b7-8259-7b41a9f14866 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcf48935946543e8a55c5df282cefe76, user_id=5a9add3165a545019bfb94874a8812e4, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:40.959413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d72474-1c08-44b7-8259-7b41a9f14866 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:40.973951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1298/2610] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:40 2022] PUT /identity/v3/projects/fcf48935946543e8a55c5df282cefe76/users/5a9add3165a545019bfb94874a8812e4/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:40.977994 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f5c24-b371-4446-a27e-56178814bdd3 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:40.988799 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f5c24-b371-4446-a27e-56178814bdd3 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:41.017542 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:41.049725 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6c4c8-6874-4c89-a5b6-c2850e3f3fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:41.050348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6c4c8-6874-4c89-a5b6-c2850e3f3fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:41.050627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6c4c8-6874-4c89-a5b6-c2850e3f3fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:41.050899 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6c4c8-6874-4c89-a5b6-c2850e3f3fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:41.051583 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6c4c8-6874-4c89-a5b6-c2850e3f3fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:41.053211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6c4c8-6874-4c89-a5b6-c2850e3f3fb1 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:41.062971 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1313/2611] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:41.067249 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d72474-1c08-44b7-8259-7b41a9f14866 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:41.079866 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d72474-1c08-44b7-8259-7b41a9f14866 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:41.084478 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b345453-42f9-40cd-a386-a90a552d76b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:41.085167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b345453-42f9-40cd-a386-a90a552d76b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:41.085452 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b345453-42f9-40cd-a386-a90a552d76b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:41.085721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b345453-42f9-40cd-a386-a90a552d76b2 admin admin] PATH_INFO: `/v3/projects/fcf48935946543e8a55c5df282cefe76/users/5a9add3165a545019bfb94874a8812e4/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:41.087660 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b345453-42f9-40cd-a386-a90a552d76b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcf48935946543e8a55c5df282cefe76, user_id=5a9add3165a545019bfb94874a8812e4, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:41.090349 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b345453-42f9-40cd-a386-a90a552d76b2 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:41.101911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1299/2612] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:41 2022] PUT /identity/v3/projects/fcf48935946543e8a55c5df282cefe76/users/5a9add3165a545019bfb94874a8812e4/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:41.113671 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfbb71-eda8-4029-81db-7d241a78ec54 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:41.113962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfbb71-eda8-4029-81db-7d241a78ec54 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:41.114247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfbb71-eda8-4029-81db-7d241a78ec54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:41.143316 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-11dfbb71-eda8-4029-81db-7d241a78ec54 None None] MFA Rules not processed for user `5a9add3165a545019bfb94874a8812e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:41.144524 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-11dfbb71-eda8-4029-81db-7d241a78ec54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:41.241011 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1314/2613] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:41 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:41.252366 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b345453-42f9-40cd-a386-a90a552d76b2 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:41.254215 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-2b345453-42f9-40cd-a386-a90a552d76b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:41.328577 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b345453-42f9-40cd-a386-a90a552d76b2 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:41.332245 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed381f41-c1b5-403a-a135-58d3f6e3a572 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:41.332805 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed381f41-c1b5-403a-a135-58d3f6e3a572 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:41.333113 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed381f41-c1b5-403a-a135-58d3f6e3a572 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:41.333390 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed381f41-c1b5-403a-a135-58d3f6e3a572 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:41.343422 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed381f41-c1b5-403a-a135-58d3f6e3a572 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:41.344855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed381f41-c1b5-403a-a135-58d3f6e3a572 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:41.354671 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1300/2614] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:41 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:42.738769 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfbb71-eda8-4029-81db-7d241a78ec54 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:42.857309 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfbb71-eda8-4029-81db-7d241a78ec54 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:42.861858 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ff3bc-e360-4590-9d1a-cfe7342ce5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:42.862499 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ff3bc-e360-4590-9d1a-cfe7342ce5f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:42.862779 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ff3bc-e360-4590-9d1a-cfe7342ce5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:42.863055 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ff3bc-e360-4590-9d1a-cfe7342ce5f7 admin admin] PATH_INFO: `/v3/users/47b4280ed1754f3d988e65a631d43d24` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:42.864233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ff3bc-e360-4590-9d1a-cfe7342ce5f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47b4280ed1754f3d988e65a631d43d24)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:42.866165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ff3bc-e360-4590-9d1a-cfe7342ce5f7 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:42.878951 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:42.915706 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-914ff3bc-e360-4590-9d1a-cfe7342ce5f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47b4280ed1754f3d988e65a631d43d24'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:42.924649 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-914ff3bc-e360-4590-9d1a-cfe7342ce5f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47b4280ed1754f3d988e65a631d43d24'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:42.947110 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-914ff3bc-e360-4590-9d1a-cfe7342ce5f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47b4280ed1754f3d988e65a631d43d24'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:42.958637 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-914ff3bc-e360-4590-9d1a-cfe7342ce5f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47b4280ed1754f3d988e65a631d43d24'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:42.959956 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-914ff3bc-e360-4590-9d1a-cfe7342ce5f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47b4280ed1754f3d988e65a631d43d24'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:42.971106 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1315/2615] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:43:42 2022] DELETE /identity/v3/users/47b4280ed1754f3d988e65a631d43d24 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:42.972948 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe437646-85fb-4835-8a67-6c1c880daede None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:42.973413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe437646-85fb-4835-8a67-6c1c880daede None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:42.973498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe437646-85fb-4835-8a67-6c1c880daede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:43.033589 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-fe437646-85fb-4835-8a67-6c1c880daede None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:43.035077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-fe437646-85fb-4835-8a67-6c1c880daede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:43.105833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1301/2616] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:43.116099 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ff3bc-e360-4590-9d1a-cfe7342ce5f7 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:43.117635 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-914ff3bc-e360-4590-9d1a-cfe7342ce5f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:43.195806 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ff3bc-e360-4590-9d1a-cfe7342ce5f7 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:43.199715 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a49ff-b274-477e-bf5b-bcc6deb980a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:43.200445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a49ff-b274-477e-bf5b-bcc6deb980a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:43.200859 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a49ff-b274-477e-bf5b-bcc6deb980a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:43.201162 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a49ff-b274-477e-bf5b-bcc6deb980a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:43.211217 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a49ff-b274-477e-bf5b-bcc6deb980a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:43.212665 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a49ff-b274-477e-bf5b-bcc6deb980a9 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:43.223826 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1316/2617] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:43.539141 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe437646-85fb-4835-8a67-6c1c880daede None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:43.540746 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-fe437646-85fb-4835-8a67-6c1c880daede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:43.561518 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe437646-85fb-4835-8a67-6c1c880daede None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:43.566531 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64801b-34ac-4ce0-81b3-18e98eadd704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:43.567343 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64801b-34ac-4ce0-81b3-18e98eadd704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:43.567495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64801b-34ac-4ce0-81b3-18e98eadd704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:43.567749 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64801b-34ac-4ce0-81b3-18e98eadd704 admin admin] PATH_INFO: `/v3/projects/732e82dff9c84034be19f702de536d69` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:43.568959 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64801b-34ac-4ce0-81b3-18e98eadd704 admin admin] RBAC: Authorizing `identity:delete_project(project_id=732e82dff9c84034be19f702de536d69)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:43.570934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64801b-34ac-4ce0-81b3-18e98eadd704 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:43.598851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:43.625126 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0a64801b-34ac-4ce0-81b3-18e98eadd704 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '732e82dff9c84034be19f702de536d69', 'request_id': 'req-0a64801b-34ac-4ce0-81b3-18e98eadd704'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:43.631160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0a64801b-34ac-4ce0-81b3-18e98eadd704 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '732e82dff9c84034be19f702de536d69', 'request_id': 'req-0a64801b-34ac-4ce0-81b3-18e98eadd704'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:43.639664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0a64801b-34ac-4ce0-81b3-18e98eadd704 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '732e82dff9c84034be19f702de536d69', 'request_id': 'req-0a64801b-34ac-4ce0-81b3-18e98eadd704'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:43.651761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0a64801b-34ac-4ce0-81b3-18e98eadd704 admin admin] The token cache is being invalidate because project 732e82dff9c84034be19f702de536d69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:43:43.652111 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0a64801b-34ac-4ce0-81b3-18e98eadd704 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:43.652771 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0a64801b-34ac-4ce0-81b3-18e98eadd704 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:43.654409 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1302/2618] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:43:43 2022] DELETE /identity/v3/projects/732e82dff9c84034be19f702de536d69 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:43:43.658424 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a49ff-b274-477e-bf5b-bcc6deb980a9 service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:43.668034 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a49ff-b274-477e-bf5b-bcc6deb980a9 service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:43.727366 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce939a87-b0eb-462d-8cf2-4a61d4efb4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:43.728033 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce939a87-b0eb-462d-8cf2-4a61d4efb4de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:43.728307 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce939a87-b0eb-462d-8cf2-4a61d4efb4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:43.728581 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce939a87-b0eb-462d-8cf2-4a61d4efb4de admin admin] PATH_INFO: `/v3/users/5a9add3165a545019bfb94874a8812e4` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:43.729801 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce939a87-b0eb-462d-8cf2-4a61d4efb4de admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a9add3165a545019bfb94874a8812e4)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:43.731893 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce939a87-b0eb-462d-8cf2-4a61d4efb4de admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:43.758662 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:43.802423 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ce939a87-b0eb-462d-8cf2-4a61d4efb4de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a9add3165a545019bfb94874a8812e4'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:43.814757 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ce939a87-b0eb-462d-8cf2-4a61d4efb4de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a9add3165a545019bfb94874a8812e4'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:43.832803 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ce939a87-b0eb-462d-8cf2-4a61d4efb4de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a9add3165a545019bfb94874a8812e4'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:43.841203 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ce939a87-b0eb-462d-8cf2-4a61d4efb4de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a9add3165a545019bfb94874a8812e4'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:43.841961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-ce939a87-b0eb-462d-8cf2-4a61d4efb4de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a9add3165a545019bfb94874a8812e4'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:43.853117 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1317/2619] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:43:43 2022] DELETE /identity/v3/users/5a9add3165a545019bfb94874a8812e4 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:44.578474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64801b-34ac-4ce0-81b3-18e98eadd704 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:44.582344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0a64801b-34ac-4ce0-81b3-18e98eadd704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:44.622264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64801b-34ac-4ce0-81b3-18e98eadd704 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:44.702334 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485557d-ec84-496c-a446-2e94b700e5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:44.702996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485557d-ec84-496c-a446-2e94b700e5b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:44.703279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485557d-ec84-496c-a446-2e94b700e5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:44.703743 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485557d-ec84-496c-a446-2e94b700e5b8 admin admin] PATH_INFO: `/v3/projects/fcf48935946543e8a55c5df282cefe76` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:44.706914 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485557d-ec84-496c-a446-2e94b700e5b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcf48935946543e8a55c5df282cefe76)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:44.708867 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485557d-ec84-496c-a446-2e94b700e5b8 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:44.778994 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-3485557d-ec84-496c-a446-2e94b700e5b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcf48935946543e8a55c5df282cefe76', 'request_id': 'req-3485557d-ec84-496c-a446-2e94b700e5b8'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:44.788352 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-3485557d-ec84-496c-a446-2e94b700e5b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcf48935946543e8a55c5df282cefe76', 'request_id': 'req-3485557d-ec84-496c-a446-2e94b700e5b8'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:44.797930 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-3485557d-ec84-496c-a446-2e94b700e5b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcf48935946543e8a55c5df282cefe76', 'request_id': 'req-3485557d-ec84-496c-a446-2e94b700e5b8'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:44.812231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-3485557d-ec84-496c-a446-2e94b700e5b8 admin admin] The token cache is being invalidate because project fcf48935946543e8a55c5df282cefe76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:43:44.812545 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-3485557d-ec84-496c-a446-2e94b700e5b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:44.813281 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-3485557d-ec84-496c-a446-2e94b700e5b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:44.816091 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1303/2620] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:43:44 2022] DELETE /identity/v3/projects/fcf48935946543e8a55c5df282cefe76 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:44.823582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068b466-8551-441e-95ec-db0f3f26356d None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:44.825396 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068b466-8551-441e-95ec-db0f3f26356d None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:44.825697 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068b466-8551-441e-95ec-db0f3f26356d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:44.882750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-9068b466-8551-441e-95ec-db0f3f26356d None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:44.884060 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-9068b466-8551-441e-95ec-db0f3f26356d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:44.996310 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1318/2621] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:43:45.004617 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb71277-7103-412f-96ba-182239deb16e None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:45.005005 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb71277-7103-412f-96ba-182239deb16e None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:45.005353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb71277-7103-412f-96ba-182239deb16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:45.035183 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-5bb71277-7103-412f-96ba-182239deb16e None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:45.039734 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-5bb71277-7103-412f-96ba-182239deb16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:45.056175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1304/2622] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:45.060165 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068b466-8551-441e-95ec-db0f3f26356d None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:45.070602 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068b466-8551-441e-95ec-db0f3f26356d None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:45.075558 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba2527-3b3e-4483-ae31-70865cc10031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:45.076448 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba2527-3b3e-4483-ae31-70865cc10031 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:45.076915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba2527-3b3e-4483-ae31-70865cc10031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:45.077347 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba2527-3b3e-4483-ae31-70865cc10031 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:45.078213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba2527-3b3e-4483-ae31-70865cc10031 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:43:45.078668 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba2527-3b3e-4483-ae31-70865cc10031 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:45.080832 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba2527-3b3e-4483-ae31-70865cc10031 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:45.101395 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb71277-7103-412f-96ba-182239deb16e None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:45.110968 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1319/2623] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:43:45.121163 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb71277-7103-412f-96ba-182239deb16e None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:45.137683 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc759b0-937a-46a7-be43-a018c27454df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:45.138801 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc759b0-937a-46a7-be43-a018c27454df admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:45.139462 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc759b0-937a-46a7-be43-a018c27454df admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:45.139879 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc759b0-937a-46a7-be43-a018c27454df admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:45.141520 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc759b0-937a-46a7-be43-a018c27454df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:45.144878 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc759b0-937a-46a7-be43-a018c27454df admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:45.192941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1305/2624] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:43:45 2022] POST /identity/v3/projects => generated 401 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:45.194174 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba2527-3b3e-4483-ae31-70865cc10031 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:45.205187 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba2527-3b3e-4483-ae31-70865cc10031 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:45.283018 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992091cc-644a-461a-bcc8-379295ac3d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:45.283660 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992091cc-644a-461a-bcc8-379295ac3d64 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:45.283942 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992091cc-644a-461a-bcc8-379295ac3d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:45.284884 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992091cc-644a-461a-bcc8-379295ac3d64 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:45.285148 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992091cc-644a-461a-bcc8-379295ac3d64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:45.287208 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992091cc-644a-461a-bcc8-379295ac3d64 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:45.295204 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:45.318053 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1320/2625] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:43:45 2022] POST /identity/v3/users => generated 301 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:43:45.319513 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc759b0-937a-46a7-be43-a018c27454df admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:45.332273 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc759b0-937a-46a7-be43-a018c27454df admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:45.337288 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa7c3b-f04d-4975-9bed-db3afdf500f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:45.337911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa7c3b-f04d-4975-9bed-db3afdf500f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:45.338196 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa7c3b-f04d-4975-9bed-db3afdf500f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:45.338474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa7c3b-f04d-4975-9bed-db3afdf500f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:45.339168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa7c3b-f04d-4975-9bed-db3afdf500f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:45.342159 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa7c3b-f04d-4975-9bed-db3afdf500f2 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:45.347801 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:45.356513 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1306/2626] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:43:45.357816 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992091cc-644a-461a-bcc8-379295ac3d64 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:45.369131 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992091cc-644a-461a-bcc8-379295ac3d64 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:45.375930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df99fc1-000d-4919-a7f4-473e26dd9de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:45.376551 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df99fc1-000d-4919-a7f4-473e26dd9de3 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:45.376830 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df99fc1-000d-4919-a7f4-473e26dd9de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:45.377639 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df99fc1-000d-4919-a7f4-473e26dd9de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:45.378069 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df99fc1-000d-4919-a7f4-473e26dd9de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:45.379851 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df99fc1-000d-4919-a7f4-473e26dd9de3 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:45.397966 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1321/2627] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:45.401779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa7c3b-f04d-4975-9bed-db3afdf500f2 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:45.412927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa7c3b-f04d-4975-9bed-db3afdf500f2 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:45.418130 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d89589-b3b8-41e6-a35d-9b5effc80759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:45.418939 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d89589-b3b8-41e6-a35d-9b5effc80759 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:45.419216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d89589-b3b8-41e6-a35d-9b5effc80759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:45.419726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d89589-b3b8-41e6-a35d-9b5effc80759 admin admin] PATH_INFO: `/v3/projects/0488356968324807b46f2090f348d818/users/8e0d83b846574faeba23b0745eb55103/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:45.437293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d89589-b3b8-41e6-a35d-9b5effc80759 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0488356968324807b46f2090f348d818, user_id=8e0d83b846574faeba23b0745eb55103, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:45.440275 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d89589-b3b8-41e6-a35d-9b5effc80759 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:45.469769 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1307/2628] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:45 2022] PUT /identity/v3/projects/0488356968324807b46f2090f348d818/users/8e0d83b846574faeba23b0745eb55103/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:45.480256 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06998361-d9d5-44cc-a5ac-a4096002f5cb None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:45.480541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06998361-d9d5-44cc-a5ac-a4096002f5cb None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:45.480817 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06998361-d9d5-44cc-a5ac-a4096002f5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:45.540553 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-06998361-d9d5-44cc-a5ac-a4096002f5cb None None] MFA Rules not processed for user `8e0d83b846574faeba23b0745eb55103`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:45.541720 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-06998361-d9d5-44cc-a5ac-a4096002f5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:45.670117 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d89589-b3b8-41e6-a35d-9b5effc80759 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:45.670548 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-74d89589-b3b8-41e6-a35d-9b5effc80759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:45.676932 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1322/2629] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:45.677975 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06998361-d9d5-44cc-a5ac-a4096002f5cb None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:45.810589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06998361-d9d5-44cc-a5ac-a4096002f5cb None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:45.826279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d89589-b3b8-41e6-a35d-9b5effc80759 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:45.828590 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315678cf-269f-4335-a37f-a4988cb6febe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:45.829467 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315678cf-269f-4335-a37f-a4988cb6febe admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:45.829857 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315678cf-269f-4335-a37f-a4988cb6febe admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:45.830394 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315678cf-269f-4335-a37f-a4988cb6febe admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:45.831354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315678cf-269f-4335-a37f-a4988cb6febe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:45.833289 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fca64-57d0-4597-b7ca-36d31d34ace2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:45.833679 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315678cf-269f-4335-a37f-a4988cb6febe admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:45.836501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fca64-57d0-4597-b7ca-36d31d34ace2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:45.837148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fca64-57d0-4597-b7ca-36d31d34ace2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:45.837600 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fca64-57d0-4597-b7ca-36d31d34ace2 admin admin] PATH_INFO: `/v3/users/40e28984712b4311bf769a93b4672536` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:45.839011 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9fca64-57d0-4597-b7ca-36d31d34ace2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40e28984712b4311bf769a93b4672536)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:45.843168 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9fca64-57d0-4597-b7ca-36d31d34ace2 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:45.916976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1323/2630] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:43:45 2022] POST /identity/v3/projects => generated 399 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:45.922463 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315678cf-269f-4335-a37f-a4988cb6febe admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:45.970494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6f9fca64-57d0-4597-b7ca-36d31d34ace2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40e28984712b4311bf769a93b4672536'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:46.010018 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6f9fca64-57d0-4597-b7ca-36d31d34ace2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40e28984712b4311bf769a93b4672536'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:46.050781 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6f9fca64-57d0-4597-b7ca-36d31d34ace2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40e28984712b4311bf769a93b4672536'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:46.067020 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6f9fca64-57d0-4597-b7ca-36d31d34ace2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40e28984712b4311bf769a93b4672536'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:46.067815 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6f9fca64-57d0-4597-b7ca-36d31d34ace2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40e28984712b4311bf769a93b4672536'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:46.077568 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1308/2631] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:43:45 2022] DELETE /identity/v3/users/40e28984712b4311bf769a93b4672536 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:46.081012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8a1b6-2656-4d9e-b240-cbe39846ee44 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:46.082665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8a1b6-2656-4d9e-b240-cbe39846ee44 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:46.082803 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8a1b6-2656-4d9e-b240-cbe39846ee44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:46.095340 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:46.114387 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315678cf-269f-4335-a37f-a4988cb6febe admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:46.120086 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-6fd8a1b6-2656-4d9e-b240-cbe39846ee44 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:46.122467 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6fd8a1b6-2656-4d9e-b240-cbe39846ee44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:46.203048 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef6258-4b80-44c4-a7ed-0988a0b2e3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:46.203392 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef6258-4b80-44c4-a7ed-0988a0b2e3bf admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:46.203693 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef6258-4b80-44c4-a7ed-0988a0b2e3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:46.204006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef6258-4b80-44c4-a7ed-0988a0b2e3bf admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:46.204720 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef6258-4b80-44c4-a7ed-0988a0b2e3bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:46.207257 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef6258-4b80-44c4-a7ed-0988a0b2e3bf admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:46.208838 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:46.225114 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1309/2632] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:46.238281 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8a1b6-2656-4d9e-b240-cbe39846ee44 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:46.241238 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6fd8a1b6-2656-4d9e-b240-cbe39846ee44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:46.245721 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1324/2633] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:43:45 2022] POST /identity/v3/users => generated 306 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:46.256869 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef6258-4b80-44c4-a7ed-0988a0b2e3bf admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:46.260582 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-8eef6258-4b80-44c4-a7ed-0988a0b2e3bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:46.279478 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef6258-4b80-44c4-a7ed-0988a0b2e3bf admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:46.287738 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7916004-9b96-417c-b79a-eaf78213ddc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:46.289102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7916004-9b96-417c-b79a-eaf78213ddc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:46.289727 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7916004-9b96-417c-b79a-eaf78213ddc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:46.290239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7916004-9b96-417c-b79a-eaf78213ddc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:46.292551 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7916004-9b96-417c-b79a-eaf78213ddc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:46.294824 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7916004-9b96-417c-b79a-eaf78213ddc9 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:46.310379 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1325/2634] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:43:46.315141 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7916004-9b96-417c-b79a-eaf78213ddc9 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:46.319885 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8a1b6-2656-4d9e-b240-cbe39846ee44 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:46.323612 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a91d4-953a-44cc-8d82-efbaa7d14412 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:46.324250 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a91d4-953a-44cc-8d82-efbaa7d14412 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:46.324538 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a91d4-953a-44cc-8d82-efbaa7d14412 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:46.325411 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a91d4-953a-44cc-8d82-efbaa7d14412 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:46.337111 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7916004-9b96-417c-b79a-eaf78213ddc9 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:46.341282 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a91d4-953a-44cc-8d82-efbaa7d14412 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:46.342760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a91d4-953a-44cc-8d82-efbaa7d14412 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:46.350069 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6d71d-6409-4a1a-9392-29eed678db9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:46.351455 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6d71d-6409-4a1a-9392-29eed678db9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:46.351920 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6d71d-6409-4a1a-9392-29eed678db9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:46.352076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6d71d-6409-4a1a-9392-29eed678db9e admin admin] PATH_INFO: `/v3/projects/3dfdec6d4cdd4012920c01dc1907c4cc/users/c257bb3b3abd430698dc4bfadda7cd21/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:46.358890 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1310/2635] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:43:46.375519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6d71d-6409-4a1a-9392-29eed678db9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dfdec6d4cdd4012920c01dc1907c4cc, user_id=c257bb3b3abd430698dc4bfadda7cd21, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:46.379590 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6d71d-6409-4a1a-9392-29eed678db9e admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:46.409026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a91d4-953a-44cc-8d82-efbaa7d14412 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:46.409663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1326/2636] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:46 2022] PUT /identity/v3/projects/3dfdec6d4cdd4012920c01dc1907c4cc/users/c257bb3b3abd430698dc4bfadda7cd21/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:46.423345 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a91d4-953a-44cc-8d82-efbaa7d14412 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:46.544601 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3a58c-6534-4ba4-98cd-94e12d0efe55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:46.547189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3a58c-6534-4ba4-98cd-94e12d0efe55 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:46.548009 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3a58c-6534-4ba4-98cd-94e12d0efe55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:46.548009 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3a58c-6534-4ba4-98cd-94e12d0efe55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:46.549321 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a3a58c-6534-4ba4-98cd-94e12d0efe55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:46.550984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a3a58c-6534-4ba4-98cd-94e12d0efe55 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:46.585830 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1311/2637] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:46.587509 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6d71d-6409-4a1a-9392-29eed678db9e admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:46.606106 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6d71d-6409-4a1a-9392-29eed678db9e admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:46.616591 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6a154-b8ba-411a-b196-f12d93e25d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:46.618714 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6a154-b8ba-411a-b196-f12d93e25d9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:46.619221 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6a154-b8ba-411a-b196-f12d93e25d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:46.620151 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6a154-b8ba-411a-b196-f12d93e25d9a admin admin] PATH_INFO: `/v3/projects/3dfdec6d4cdd4012920c01dc1907c4cc/users/c257bb3b3abd430698dc4bfadda7cd21/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:46.624491 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6a154-b8ba-411a-b196-f12d93e25d9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dfdec6d4cdd4012920c01dc1907c4cc, user_id=c257bb3b3abd430698dc4bfadda7cd21, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:46.628597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6a154-b8ba-411a-b196-f12d93e25d9a admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:46.682382 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1327/2638] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:46 2022] PUT /identity/v3/projects/3dfdec6d4cdd4012920c01dc1907c4cc/users/c257bb3b3abd430698dc4bfadda7cd21/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:43:46.694988 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07915301-a551-4f0a-be43-769af5f9a5c5 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:46.695938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07915301-a551-4f0a-be43-769af5f9a5c5 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:46.696568 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07915301-a551-4f0a-be43-769af5f9a5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:46.769221 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-07915301-a551-4f0a-be43-769af5f9a5c5 None None] MFA Rules not processed for user `c257bb3b3abd430698dc4bfadda7cd21`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:46.770611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-07915301-a551-4f0a-be43-769af5f9a5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:46.934912 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1312/2639] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:46 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:46.948023 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6a154-b8ba-411a-b196-f12d93e25d9a admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:46.954150 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7bb6a154-b8ba-411a-b196-f12d93e25d9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:47.006946 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07915301-a551-4f0a-be43-769af5f9a5c5 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:47.008422 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-07915301-a551-4f0a-be43-769af5f9a5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:47.059481 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:47.073502 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6a154-b8ba-411a-b196-f12d93e25d9a admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:47.080663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e545c-5cbc-4602-9034-71dd6505f0de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:47.081577 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e545c-5cbc-4602-9034-71dd6505f0de service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:47.082055 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e545c-5cbc-4602-9034-71dd6505f0de service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:47.082483 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e545c-5cbc-4602-9034-71dd6505f0de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:47.098328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e545c-5cbc-4602-9034-71dd6505f0de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:47.102775 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e545c-5cbc-4602-9034-71dd6505f0de service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:47.136538 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1328/2640] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:46 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:43:47.158578 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07915301-a551-4f0a-be43-769af5f9a5c5 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:47.174947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1035f-488a-4b19-8830-8beefbcb1827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:47.175668 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1035f-488a-4b19-8830-8beefbcb1827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:47.175956 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1035f-488a-4b19-8830-8beefbcb1827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:47.180309 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1035f-488a-4b19-8830-8beefbcb1827 admin admin] PATH_INFO: `/v3/projects/c93a0e8d8e004bd4bacc67363ac5a9bc` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:47.187677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b1035f-488a-4b19-8830-8beefbcb1827 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c93a0e8d8e004bd4bacc67363ac5a9bc)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:47.189695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b1035f-488a-4b19-8830-8beefbcb1827 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:47.271009 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e545c-5cbc-4602-9034-71dd6505f0de service cinder] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:47.272553 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-074e545c-5cbc-4602-9034-71dd6505f0de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:47.285449 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b5b1035f-488a-4b19-8830-8beefbcb1827 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c93a0e8d8e004bd4bacc67363ac5a9bc', 'request_id': 'req-b5b1035f-488a-4b19-8830-8beefbcb1827'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:47.289110 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e545c-5cbc-4602-9034-71dd6505f0de service cinder] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:47.293391 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b5b1035f-488a-4b19-8830-8beefbcb1827 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c93a0e8d8e004bd4bacc67363ac5a9bc', 'request_id': 'req-b5b1035f-488a-4b19-8830-8beefbcb1827'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:47.338350 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b5b1035f-488a-4b19-8830-8beefbcb1827 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c93a0e8d8e004bd4bacc67363ac5a9bc', 'request_id': 'req-b5b1035f-488a-4b19-8830-8beefbcb1827'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:47.357558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b5b1035f-488a-4b19-8830-8beefbcb1827 admin admin] The token cache is being invalidate because project c93a0e8d8e004bd4bacc67363ac5a9bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:43:47.358045 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b5b1035f-488a-4b19-8830-8beefbcb1827 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:47.358771 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-b5b1035f-488a-4b19-8830-8beefbcb1827 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:47.361181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1313/2641] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:43:47 2022] DELETE /identity/v3/projects/c93a0e8d8e004bd4bacc67363ac5a9bc => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:43:47.369076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:47.370786 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16771bd4-0a17-49a1-815c-e93c5a3f8e02 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:47.371424 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16771bd4-0a17-49a1-815c-e93c5a3f8e02 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:47.372482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16771bd4-0a17-49a1-815c-e93c5a3f8e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:47.388188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cb066-bd2c-4f15-b6e6-7994cbd6403d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:47.389259 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cb066-bd2c-4f15-b6e6-7994cbd6403d service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:47.389802 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cb066-bd2c-4f15-b6e6-7994cbd6403d service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:47.390314 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cb066-bd2c-4f15-b6e6-7994cbd6403d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:47.392785 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-549cb066-bd2c-4f15-b6e6-7994cbd6403d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:47.401835 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-16771bd4-0a17-49a1-815c-e93c5a3f8e02 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:47.404193 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-16771bd4-0a17-49a1-815c-e93c5a3f8e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:47.500781 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1314/2642] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:47.509167 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210afdb8-7a05-458c-b56b-f2badfaa8cae None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:47.509534 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210afdb8-7a05-458c-b56b-f2badfaa8cae None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:47.510272 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210afdb8-7a05-458c-b56b-f2badfaa8cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:47.518698 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549cb066-bd2c-4f15-b6e6-7994cbd6403d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:47.521667 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549cb066-bd2c-4f15-b6e6-7994cbd6403d service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:47.541812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-210afdb8-7a05-458c-b56b-f2badfaa8cae None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:47.543104 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-210afdb8-7a05-458c-b56b-f2badfaa8cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:47.545715 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1329/2643] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:43:47.556965 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1315/2644] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:47.562625 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cb066-bd2c-4f15-b6e6-7994cbd6403d service cinder] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:47.575984 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cb066-bd2c-4f15-b6e6-7994cbd6403d service cinder] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:47.581158 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc181fe-5a1e-446b-a51f-c81837ff5d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:47.582474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc181fe-5a1e-446b-a51f-c81837ff5d79 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:47.582769 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc181fe-5a1e-446b-a51f-c81837ff5d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:47.583050 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc181fe-5a1e-446b-a51f-c81837ff5d79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:47.583814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc181fe-5a1e-446b-a51f-c81837ff5d79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:43:47.584127 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc181fe-5a1e-446b-a51f-c81837ff5d79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:47.585897 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc181fe-5a1e-446b-a51f-c81837ff5d79 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:47.605100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1330/2645] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:43:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:43:47.609637 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210afdb8-7a05-458c-b56b-f2badfaa8cae None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:47.623555 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210afdb8-7a05-458c-b56b-f2badfaa8cae None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:47.630584 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613c236-5e49-42bf-bbbb-743de21cbf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:47.631214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613c236-5e49-42bf-bbbb-743de21cbf13 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:47.631535 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613c236-5e49-42bf-bbbb-743de21cbf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:47.631777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613c236-5e49-42bf-bbbb-743de21cbf13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:47.632504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b613c236-5e49-42bf-bbbb-743de21cbf13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:47.634570 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b613c236-5e49-42bf-bbbb-743de21cbf13 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:47.669465 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1316/2646] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:43:47 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:47.676803 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc181fe-5a1e-446b-a51f-c81837ff5d79 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:47.693340 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc181fe-5a1e-446b-a51f-c81837ff5d79 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:47.757026 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5648e-b349-45d9-ad73-94b61aaf058e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:47.757682 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5648e-b349-45d9-ad73-94b61aaf058e admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:47.757982 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5648e-b349-45d9-ad73-94b61aaf058e admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:47.758259 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5648e-b349-45d9-ad73-94b61aaf058e admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:47.758922 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5648e-b349-45d9-ad73-94b61aaf058e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:47.760839 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5648e-b349-45d9-ad73-94b61aaf058e admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:47.783390 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1331/2647] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:43:47 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:47.788202 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613c236-5e49-42bf-bbbb-743de21cbf13 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:47.799474 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613c236-5e49-42bf-bbbb-743de21cbf13 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:47.804574 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28567dd5-71cb-4260-97ba-b80e17b5fde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:47.805222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28567dd5-71cb-4260-97ba-b80e17b5fde9 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:47.816947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28567dd5-71cb-4260-97ba-b80e17b5fde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:47.816947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28567dd5-71cb-4260-97ba-b80e17b5fde9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:47.816947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28567dd5-71cb-4260-97ba-b80e17b5fde9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:47.816947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28567dd5-71cb-4260-97ba-b80e17b5fde9 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:47.828158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1317/2648] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:47.832344 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5648e-b349-45d9-ad73-94b61aaf058e admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:47.842816 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5648e-b349-45d9-ad73-94b61aaf058e admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:47.850788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48538705-fdc9-4c3e-a221-a9319240f6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:47.850788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48538705-fdc9-4c3e-a221-a9319240f6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:47.850788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48538705-fdc9-4c3e-a221-a9319240f6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:47.850788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48538705-fdc9-4c3e-a221-a9319240f6d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:47.851312 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48538705-fdc9-4c3e-a221-a9319240f6d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:47.851790 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48538705-fdc9-4c3e-a221-a9319240f6d0 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:47.868746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1332/2649] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:47.873395 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28567dd5-71cb-4260-97ba-b80e17b5fde9 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:47.886306 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28567dd5-71cb-4260-97ba-b80e17b5fde9 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:47.891656 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90931356-d7d0-46b2-9877-1d687219e471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:47.892401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90931356-d7d0-46b2-9877-1d687219e471 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:47.892695 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90931356-d7d0-46b2-9877-1d687219e471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:47.893002 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90931356-d7d0-46b2-9877-1d687219e471 admin admin] PATH_INFO: `/v3/projects/76552deb302841cc86b960fe1f50b2c4/users/73a4e537be0d4c88bcfd55c6adb2dd38/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:47.901810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:47.910434 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90931356-d7d0-46b2-9877-1d687219e471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76552deb302841cc86b960fe1f50b2c4, user_id=73a4e537be0d4c88bcfd55c6adb2dd38, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:47.913565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90931356-d7d0-46b2-9877-1d687219e471 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:47.927327 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1318/2650] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:47 2022] PUT /identity/v3/projects/76552deb302841cc86b960fe1f50b2c4/users/73a4e537be0d4c88bcfd55c6adb2dd38/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:43:47.938432 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6f3f7-feea-419d-8e45-5f9152819bf7 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:47.938718 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6f3f7-feea-419d-8e45-5f9152819bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:47.939041 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6f3f7-feea-419d-8e45-5f9152819bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:47.967682 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-91f6f3f7-feea-419d-8e45-5f9152819bf7 None None] MFA Rules not processed for user `73a4e537be0d4c88bcfd55c6adb2dd38`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:47.968909 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-91f6f3f7-feea-419d-8e45-5f9152819bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:48.024935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:48.037598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1333/2651] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:48.049483 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90931356-d7d0-46b2-9877-1d687219e471 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:48.051364 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-90931356-d7d0-46b2-9877-1d687219e471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:48.158164 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90931356-d7d0-46b2-9877-1d687219e471 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:48.163971 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e67c6-567a-4f1f-b586-34497418ecdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:48.164577 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e67c6-567a-4f1f-b586-34497418ecdd service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:48.164862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e67c6-567a-4f1f-b586-34497418ecdd service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:48.165163 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e67c6-567a-4f1f-b586-34497418ecdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:48.174213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e67c6-567a-4f1f-b586-34497418ecdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:48.175865 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e67c6-567a-4f1f-b586-34497418ecdd service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:48.184856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1319/2652] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:48.395007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6f3f7-feea-419d-8e45-5f9152819bf7 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:48.461819 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6f3f7-feea-419d-8e45-5f9152819bf7 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:48.466385 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b601d3a-4678-4346-8b6b-08473cfd0a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:48.467029 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b601d3a-4678-4346-8b6b-08473cfd0a2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:48.467309 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b601d3a-4678-4346-8b6b-08473cfd0a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:48.467580 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b601d3a-4678-4346-8b6b-08473cfd0a2f admin admin] PATH_INFO: `/v3/users/73a4e537be0d4c88bcfd55c6adb2dd38` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:48.468737 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b601d3a-4678-4346-8b6b-08473cfd0a2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=73a4e537be0d4c88bcfd55c6adb2dd38)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:48.470906 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b601d3a-4678-4346-8b6b-08473cfd0a2f admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:48.492022 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d4268-b957-4720-8e89-b5c2f9523c9e None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:48.492310 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d4268-b957-4720-8e89-b5c2f9523c9e None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:48.492583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d4268-b957-4720-8e89-b5c2f9523c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:48.512733 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-ba9d4268-b957-4720-8e89-b5c2f9523c9e None None] MFA Rules not processed for user `f965c9163ebf41a9866af7389feaf17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:48.514005 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ba9d4268-b957-4720-8e89-b5c2f9523c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:48.521447 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6b601d3a-4678-4346-8b6b-08473cfd0a2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73a4e537be0d4c88bcfd55c6adb2dd38'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:48.529601 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6b601d3a-4678-4346-8b6b-08473cfd0a2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73a4e537be0d4c88bcfd55c6adb2dd38'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:48.542559 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6b601d3a-4678-4346-8b6b-08473cfd0a2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73a4e537be0d4c88bcfd55c6adb2dd38'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:48.550328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6b601d3a-4678-4346-8b6b-08473cfd0a2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73a4e537be0d4c88bcfd55c6adb2dd38'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:48.551166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6b601d3a-4678-4346-8b6b-08473cfd0a2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73a4e537be0d4c88bcfd55c6adb2dd38'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:48.561574 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1334/2653] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:43:48 2022] DELETE /identity/v3/users/73a4e537be0d4c88bcfd55c6adb2dd38 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:48.563219 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b2beb-d53b-4c32-bf01-1b65acd92fbe None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:48.565190 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b2beb-d53b-4c32-bf01-1b65acd92fbe None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:48.565808 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b2beb-d53b-4c32-bf01-1b65acd92fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:48.578548 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1320/2654] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:43:48.600631 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-848b2beb-d53b-4c32-bf01-1b65acd92fbe None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:48.602913 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-848b2beb-d53b-4c32-bf01-1b65acd92fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:48.698168 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1335/2655] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:43:48.709844 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d4268-b957-4720-8e89-b5c2f9523c9e None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:48.712155 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ba9d4268-b957-4720-8e89-b5c2f9523c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:48.788610 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d4268-b957-4720-8e89-b5c2f9523c9e None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:48.792369 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:48.838514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94471225-e79a-4fc1-9657-3b739642652f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:48.839024 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94471225-e79a-4fc1-9657-3b739642652f service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:48.839305 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94471225-e79a-4fc1-9657-3b739642652f service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:48.839582 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94471225-e79a-4fc1-9657-3b739642652f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:48.849535 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94471225-e79a-4fc1-9657-3b739642652f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:48.850974 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94471225-e79a-4fc1-9657-3b739642652f service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:48.861135 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1321/2656] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:49.105316 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b2beb-d53b-4c32-bf01-1b65acd92fbe None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:49.107812 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-848b2beb-d53b-4c32-bf01-1b65acd92fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:49.123307 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b2beb-d53b-4c32-bf01-1b65acd92fbe None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:49.129077 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a922144-3339-4507-828a-9c2ed0e98eb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:49.130028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a922144-3339-4507-828a-9c2ed0e98eb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:49.130722 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a922144-3339-4507-828a-9c2ed0e98eb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:49.131007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a922144-3339-4507-828a-9c2ed0e98eb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:49.133282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7a922144-3339-4507-828a-9c2ed0e98eb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:49.145282 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a922144-3339-4507-828a-9c2ed0e98eb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:49.146722 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a922144-3339-4507-828a-9c2ed0e98eb0 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:49.151689 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:49.157156 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1336/2657] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:49.210778 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94471225-e79a-4fc1-9657-3b739642652f service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:49.212536 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-94471225-e79a-4fc1-9657-3b739642652f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:49.236022 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94471225-e79a-4fc1-9657-3b739642652f service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:49.242759 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea85bc-9494-4633-9fed-9d85012daaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:49.243420 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea85bc-9494-4633-9fed-9d85012daaec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:49.243809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea85bc-9494-4633-9fed-9d85012daaec admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:49.243981 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea85bc-9494-4633-9fed-9d85012daaec admin admin] PATH_INFO: `/v3/projects/76552deb302841cc86b960fe1f50b2c4` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:49.247686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea85bc-9494-4633-9fed-9d85012daaec admin admin] RBAC: Authorizing `identity:delete_project(project_id=76552deb302841cc86b960fe1f50b2c4)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:49.250058 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea85bc-9494-4633-9fed-9d85012daaec admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:49.331239 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6fea85bc-9494-4633-9fed-9d85012daaec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76552deb302841cc86b960fe1f50b2c4', 'request_id': 'req-6fea85bc-9494-4633-9fed-9d85012daaec'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:49.339352 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6fea85bc-9494-4633-9fed-9d85012daaec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76552deb302841cc86b960fe1f50b2c4', 'request_id': 'req-6fea85bc-9494-4633-9fed-9d85012daaec'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:49.354996 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6fea85bc-9494-4633-9fed-9d85012daaec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76552deb302841cc86b960fe1f50b2c4', 'request_id': 'req-6fea85bc-9494-4633-9fed-9d85012daaec'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:49.373900 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6fea85bc-9494-4633-9fed-9d85012daaec admin admin] The token cache is being invalidate because project 76552deb302841cc86b960fe1f50b2c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:43:49.374355 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6fea85bc-9494-4633-9fed-9d85012daaec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:49.375078 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6fea85bc-9494-4633-9fed-9d85012daaec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:49.376835 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1322/2658] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:43:49 2022] DELETE /identity/v3/projects/76552deb302841cc86b960fe1f50b2c4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:49.382284 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000610e-cbe5-4f9c-b2e4-88812d4b7166 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:49.382593 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000610e-cbe5-4f9c-b2e4-88812d4b7166 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:49.382946 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000610e-cbe5-4f9c-b2e4-88812d4b7166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:49.402807 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-7000610e-cbe5-4f9c-b2e4-88812d4b7166 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:49.404104 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7000610e-cbe5-4f9c-b2e4-88812d4b7166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:49.472956 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1337/2659] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:43:49.476966 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4116b-2b2e-41cc-9031-ce5de3eee255 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:49.477451 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4116b-2b2e-41cc-9031-ce5de3eee255 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:49.477732 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4116b-2b2e-41cc-9031-ce5de3eee255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:49.499252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-8ae4116b-2b2e-41cc-9031-ce5de3eee255 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:49.500469 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-8ae4116b-2b2e-41cc-9031-ce5de3eee255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:49.516231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1323/2660] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:43:49.517460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000610e-cbe5-4f9c-b2e4-88812d4b7166 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:49.528186 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000610e-cbe5-4f9c-b2e4-88812d4b7166 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:49.535209 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f2f00-e709-439e-889f-c3d69f7886c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:49.535847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f2f00-e709-439e-889f-c3d69f7886c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:49.536285 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f2f00-e709-439e-889f-c3d69f7886c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:49.536460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f2f00-e709-439e-889f-c3d69f7886c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:49.537094 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f2f00-e709-439e-889f-c3d69f7886c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:43:49.537391 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f2f00-e709-439e-889f-c3d69f7886c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:49.538981 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f2f00-e709-439e-889f-c3d69f7886c2 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:49.556350 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1338/2661] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:43:49.557454 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4116b-2b2e-41cc-9031-ce5de3eee255 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:49.568881 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4116b-2b2e-41cc-9031-ce5de3eee255 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:49.573847 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4619a-3691-46d1-b68a-12f52745c9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:49.574677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4619a-3691-46d1-b68a-12f52745c9d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:49.575999 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4619a-3691-46d1-b68a-12f52745c9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:49.575999 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4619a-3691-46d1-b68a-12f52745c9d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:49.576596 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4619a-3691-46d1-b68a-12f52745c9d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:49.578488 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4619a-3691-46d1-b68a-12f52745c9d1 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:49.616790 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1324/2662] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:43:49 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:49.618034 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f2f00-e709-439e-889f-c3d69f7886c2 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:49.628796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f2f00-e709-439e-889f-c3d69f7886c2 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:49.702752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618aad5-604a-4255-af1a-4f9035e1f145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:49.703434 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618aad5-604a-4255-af1a-4f9035e1f145 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:49.703784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618aad5-604a-4255-af1a-4f9035e1f145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:49.704072 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618aad5-604a-4255-af1a-4f9035e1f145 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:49.704765 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1618aad5-604a-4255-af1a-4f9035e1f145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:49.707044 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1618aad5-604a-4255-af1a-4f9035e1f145 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:49.745808 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1339/2663] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:43:49 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:49.750443 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4619a-3691-46d1-b68a-12f52745c9d1 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:49.765187 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4619a-3691-46d1-b68a-12f52745c9d1 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:49.771020 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ec3c0-5aa9-4d7c-a732-1876f3772e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:49.771658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ec3c0-5aa9-4d7c-a732-1876f3772e95 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:49.771945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ec3c0-5aa9-4d7c-a732-1876f3772e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:49.772222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ec3c0-5aa9-4d7c-a732-1876f3772e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:49.773080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ec3c0-5aa9-4d7c-a732-1876f3772e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:49.774910 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ec3c0-5aa9-4d7c-a732-1876f3772e95 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:49.795000 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1325/2664] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:43:49.798665 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618aad5-604a-4255-af1a-4f9035e1f145 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:49.810603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618aad5-604a-4255-af1a-4f9035e1f145 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:49.826421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925e3d7-ac90-471d-a41f-25c516f436f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:49.827639 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925e3d7-ac90-471d-a41f-25c516f436f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:49.827998 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925e3d7-ac90-471d-a41f-25c516f436f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:49.829422 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925e3d7-ac90-471d-a41f-25c516f436f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:49.830213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4925e3d7-ac90-471d-a41f-25c516f436f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:49.832440 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4925e3d7-ac90-471d-a41f-25c516f436f6 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:49.848729 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1340/2665] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:49.850059 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ec3c0-5aa9-4d7c-a732-1876f3772e95 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:49.861727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ec3c0-5aa9-4d7c-a732-1876f3772e95 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:49.866756 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce383e5d-2f19-4de9-8f47-d0f2cc74b180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:49.867426 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce383e5d-2f19-4de9-8f47-d0f2cc74b180 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:49.867709 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce383e5d-2f19-4de9-8f47-d0f2cc74b180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:49.867989 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce383e5d-2f19-4de9-8f47-d0f2cc74b180 admin admin] PATH_INFO: `/v3/projects/8c2d335bc9f6477d8909c3534fb3fbe9/users/9d2866c98ee749578ef81452bed22352/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:49.883308 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce383e5d-2f19-4de9-8f47-d0f2cc74b180 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c2d335bc9f6477d8909c3534fb3fbe9, user_id=9d2866c98ee749578ef81452bed22352, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:49.886009 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce383e5d-2f19-4de9-8f47-d0f2cc74b180 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:49.899465 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1326/2666] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:49 2022] PUT /identity/v3/projects/8c2d335bc9f6477d8909c3534fb3fbe9/users/9d2866c98ee749578ef81452bed22352/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:49.910931 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3643e-d99c-4632-947f-6a24334f70f1 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:49.911242 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3643e-d99c-4632-947f-6a24334f70f1 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:49.911515 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3643e-d99c-4632-947f-6a24334f70f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:49.943017 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-e5e3643e-d99c-4632-947f-6a24334f70f1 None None] MFA Rules not processed for user `9d2866c98ee749578ef81452bed22352`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:49.944144 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e5e3643e-d99c-4632-947f-6a24334f70f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:50.004727 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:50.016125 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1341/2667] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:50.020362 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce383e5d-2f19-4de9-8f47-d0f2cc74b180 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:50.060585 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:50.103532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce383e5d-2f19-4de9-8f47-d0f2cc74b180 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:50.109441 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a8d4b-4e91-4c5f-960a-a9587fd94eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:50.110639 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a8d4b-4e91-4c5f-960a-a9587fd94eb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:50.110929 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a8d4b-4e91-4c5f-960a-a9587fd94eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:50.111210 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a8d4b-4e91-4c5f-960a-a9587fd94eb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:50.111892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a8d4b-4e91-4c5f-960a-a9587fd94eb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:50.114244 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a8d4b-4e91-4c5f-960a-a9587fd94eb4 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:50.156576 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1327/2668] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:43:50 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:50.164319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3643e-d99c-4632-947f-6a24334f70f1 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:50.187169 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3643e-d99c-4632-947f-6a24334f70f1 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:50.257739 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0131f8a-7ab2-4734-823b-2585ef2b2c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:50.258483 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0131f8a-7ab2-4734-823b-2585ef2b2c5b admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:50.258852 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0131f8a-7ab2-4734-823b-2585ef2b2c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:50.259196 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0131f8a-7ab2-4734-823b-2585ef2b2c5b admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:50.259923 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0131f8a-7ab2-4734-823b-2585ef2b2c5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:50.261966 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0131f8a-7ab2-4734-823b-2585ef2b2c5b admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:50.283687 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1342/2669] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:43:50 2022] POST /identity/v3/users => generated 299 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:50.287803 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a8d4b-4e91-4c5f-960a-a9587fd94eb4 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:50.298986 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a8d4b-4e91-4c5f-960a-a9587fd94eb4 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:50.305126 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78de0e9-ee77-4367-9c21-4a83d08efa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:50.305840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78de0e9-ee77-4367-9c21-4a83d08efa1e admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:50.306184 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78de0e9-ee77-4367-9c21-4a83d08efa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:50.306602 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78de0e9-ee77-4367-9c21-4a83d08efa1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:50.308868 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78de0e9-ee77-4367-9c21-4a83d08efa1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:50.311160 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78de0e9-ee77-4367-9c21-4a83d08efa1e admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:50.327985 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1328/2670] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:50.332094 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0131f8a-7ab2-4734-823b-2585ef2b2c5b admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:50.342392 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0131f8a-7ab2-4734-823b-2585ef2b2c5b admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:50.348254 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028873bd-4382-44c6-9e75-b63d405b3998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:50.348960 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028873bd-4382-44c6-9e75-b63d405b3998 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:50.349245 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028873bd-4382-44c6-9e75-b63d405b3998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:50.349523 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028873bd-4382-44c6-9e75-b63d405b3998 admin admin] PATH_INFO: `/v3/projects/a15ec38be3aa4c29991b007b0e5f7566/users/fe5c9f11b6ba469183cc57d37874ece1/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:50.366983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028873bd-4382-44c6-9e75-b63d405b3998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a15ec38be3aa4c29991b007b0e5f7566, user_id=fe5c9f11b6ba469183cc57d37874ece1, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:50.369676 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028873bd-4382-44c6-9e75-b63d405b3998 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:50.388406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1343/2671] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:50 2022] PUT /identity/v3/projects/a15ec38be3aa4c29991b007b0e5f7566/users/fe5c9f11b6ba469183cc57d37874ece1/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:50.392181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78de0e9-ee77-4367-9c21-4a83d08efa1e admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:50.416592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78de0e9-ee77-4367-9c21-4a83d08efa1e admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:50.482447 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7802f-5cc9-49df-81f9-65181832f204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:50.483301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7802f-5cc9-49df-81f9-65181832f204 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:50.484956 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7802f-5cc9-49df-81f9-65181832f204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:50.485116 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7802f-5cc9-49df-81f9-65181832f204 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:50.485823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7802f-5cc9-49df-81f9-65181832f204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:50.487509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7802f-5cc9-49df-81f9-65181832f204 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:50.504947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1329/2672] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:50.507756 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028873bd-4382-44c6-9e75-b63d405b3998 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:50.520726 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028873bd-4382-44c6-9e75-b63d405b3998 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:50.525651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184eb98e-4bef-4ed9-b147-51d347029d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:50.526398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184eb98e-4bef-4ed9-b147-51d347029d70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:50.526755 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184eb98e-4bef-4ed9-b147-51d347029d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:50.527214 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184eb98e-4bef-4ed9-b147-51d347029d70 admin admin] PATH_INFO: `/v3/projects/a15ec38be3aa4c29991b007b0e5f7566/users/fe5c9f11b6ba469183cc57d37874ece1/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:50.531130 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184eb98e-4bef-4ed9-b147-51d347029d70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a15ec38be3aa4c29991b007b0e5f7566, user_id=fe5c9f11b6ba469183cc57d37874ece1, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:50.534568 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184eb98e-4bef-4ed9-b147-51d347029d70 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:50.548268 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1344/2673] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:50 2022] PUT /identity/v3/projects/a15ec38be3aa4c29991b007b0e5f7566/users/fe5c9f11b6ba469183cc57d37874ece1/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:50.560858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6b05f-fe50-499a-bd26-d7e0991ff7e8 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:50.561443 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6b05f-fe50-499a-bd26-d7e0991ff7e8 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:50.561940 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6b05f-fe50-499a-bd26-d7e0991ff7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:50.600052 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-1db6b05f-fe50-499a-bd26-d7e0991ff7e8 None None] MFA Rules not processed for user `fe5c9f11b6ba469183cc57d37874ece1`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:50.601980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-1db6b05f-fe50-499a-bd26-d7e0991ff7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:50.702263 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1330/2674] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:50 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:43:50.713231 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184eb98e-4bef-4ed9-b147-51d347029d70 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:50.715090 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-184eb98e-4bef-4ed9-b147-51d347029d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:50.796554 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184eb98e-4bef-4ed9-b147-51d347029d70 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:50.801378 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d00c55-7517-4a3f-9974-f95c23974af0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:50.802799 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d00c55-7517-4a3f-9974-f95c23974af0 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:50.803607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d00c55-7517-4a3f-9974-f95c23974af0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:50.804104 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d00c55-7517-4a3f-9974-f95c23974af0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:50.824247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d00c55-7517-4a3f-9974-f95c23974af0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:50.826008 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d00c55-7517-4a3f-9974-f95c23974af0 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:50.850503 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1345/2675] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:51.664495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6b05f-fe50-499a-bd26-d7e0991ff7e8 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:51.723336 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:51.864109 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6b05f-fe50-499a-bd26-d7e0991ff7e8 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:51.898847 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189aef1-cc21-407f-aaa9-d84acccf004b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:51.899452 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189aef1-cc21-407f-aaa9-d84acccf004b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:51.899816 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189aef1-cc21-407f-aaa9-d84acccf004b admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:51.900012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189aef1-cc21-407f-aaa9-d84acccf004b admin admin] PATH_INFO: `/v3/users/9d2866c98ee749578ef81452bed22352` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:51.906420 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f189aef1-cc21-407f-aaa9-d84acccf004b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d2866c98ee749578ef81452bed22352)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:51.908530 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f189aef1-cc21-407f-aaa9-d84acccf004b admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:52.058908 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d00c55-7517-4a3f-9974-f95c23974af0 service cinder] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:52.063208 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e3d00c55-7517-4a3f-9974-f95c23974af0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:52.069059 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-f189aef1-cc21-407f-aaa9-d84acccf004b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d2866c98ee749578ef81452bed22352'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:52.082018 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-f189aef1-cc21-407f-aaa9-d84acccf004b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d2866c98ee749578ef81452bed22352'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:52.094597 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-f189aef1-cc21-407f-aaa9-d84acccf004b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d2866c98ee749578ef81452bed22352'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:52.107522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d00c55-7517-4a3f-9974-f95c23974af0 service cinder] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:52.113850 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-f189aef1-cc21-407f-aaa9-d84acccf004b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d2866c98ee749578ef81452bed22352'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:52.114647 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-f189aef1-cc21-407f-aaa9-d84acccf004b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d2866c98ee749578ef81452bed22352'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:52.125130 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1331/2676] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:43:51 2022] DELETE /identity/v3/users/9d2866c98ee749578ef81452bed22352 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:43:52.125751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ed9d3-3d69-4369-bb47-e9dddc77411b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:52.126567 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ed9d3-3d69-4369-bb47-e9dddc77411b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:52.126860 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ed9d3-3d69-4369-bb47-e9dddc77411b admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:52.127027 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0fa24-f9c0-46fa-b8f5-fd4a1f3dfb12 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:52.127144 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ed9d3-3d69-4369-bb47-e9dddc77411b admin admin] PATH_INFO: `/v3/users/8e0d83b846574faeba23b0745eb55103` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:52.127332 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0fa24-f9c0-46fa-b8f5-fd4a1f3dfb12 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:52.127610 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0fa24-f9c0-46fa-b8f5-fd4a1f3dfb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:52.129428 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ed9d3-3d69-4369-bb47-e9dddc77411b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e0d83b846574faeba23b0745eb55103)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:52.131374 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ed9d3-3d69-4369-bb47-e9dddc77411b admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:52.162538 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-b6b0fa24-f9c0-46fa-b8f5-fd4a1f3dfb12 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:52.171288 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-b6b0fa24-f9c0-46fa-b8f5-fd4a1f3dfb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:52.253031 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:52.254962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-727ed9d3-3d69-4369-bb47-e9dddc77411b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e0d83b846574faeba23b0745eb55103'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:52.270352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-727ed9d3-3d69-4369-bb47-e9dddc77411b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e0d83b846574faeba23b0745eb55103'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:52.293534 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-727ed9d3-3d69-4369-bb47-e9dddc77411b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e0d83b846574faeba23b0745eb55103'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:52.304947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1332/2677] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:52.310614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-727ed9d3-3d69-4369-bb47-e9dddc77411b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e0d83b846574faeba23b0745eb55103'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:52.311361 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-727ed9d3-3d69-4369-bb47-e9dddc77411b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e0d83b846574faeba23b0745eb55103'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:52.323449 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02df40b-3415-493e-a073-f425ef6f5e5c None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:52.323745 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02df40b-3415-493e-a073-f425ef6f5e5c None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:52.324026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02df40b-3415-493e-a073-f425ef6f5e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:52.327429 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1346/2678] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:43:52 2022] DELETE /identity/v3/users/8e0d83b846574faeba23b0745eb55103 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:52.344788 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ed9d3-3d69-4369-bb47-e9dddc77411b admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:52.346526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-727ed9d3-3d69-4369-bb47-e9dddc77411b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:52.356879 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-d02df40b-3415-493e-a073-f425ef6f5e5c None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:52.358173 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d02df40b-3415-493e-a073-f425ef6f5e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:52.424969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:52.437033 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ed9d3-3d69-4369-bb47-e9dddc77411b admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:52.442234 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97ade8-7445-43bc-9fad-c44c00c97f33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:52.443064 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97ade8-7445-43bc-9fad-c44c00c97f33 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:52.443751 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97ade8-7445-43bc-9fad-c44c00c97f33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:52.444185 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97ade8-7445-43bc-9fad-c44c00c97f33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:52.446242 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-af97ade8-7445-43bc-9fad-c44c00c97f33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:52.454375 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1333/2679] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:52.466232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97ade8-7445-43bc-9fad-c44c00c97f33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:52.467409 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02df40b-3415-493e-a073-f425ef6f5e5c None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:52.468191 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97ade8-7445-43bc-9fad-c44c00c97f33 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:52.469016 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d02df40b-3415-493e-a073-f425ef6f5e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:52.482577 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02df40b-3415-493e-a073-f425ef6f5e5c None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:52.486781 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd2a52-f1ac-4d4e-aefa-b8887cd008a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:52.487501 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1347/2680] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:52.487634 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd2a52-f1ac-4d4e-aefa-b8887cd008a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:52.488024 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd2a52-f1ac-4d4e-aefa-b8887cd008a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:52.488366 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd2a52-f1ac-4d4e-aefa-b8887cd008a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:52.500131 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd2a52-f1ac-4d4e-aefa-b8887cd008a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:52.501788 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd2a52-f1ac-4d4e-aefa-b8887cd008a1 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:52.514508 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1334/2681] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:43:52.606510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e859b-1bea-4e78-95a6-afd5e8415e4e None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:52.607071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e859b-1bea-4e78-95a6-afd5e8415e4e None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:52.607522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e859b-1bea-4e78-95a6-afd5e8415e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:52.656373 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-a14e859b-1bea-4e78-95a6-afd5e8415e4e None None] MFA Rules not processed for user `07d6bffc69674103b4c6540598a780b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:52.660521 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-a14e859b-1bea-4e78-95a6-afd5e8415e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:52.743896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1348/2682] 10.222.0.74 () {58 vars in 967 bytes} [Wed Jul 27 16:43:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:52.801986 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd2a52-f1ac-4d4e-aefa-b8887cd008a1 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:52.803440 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-4bbd2a52-f1ac-4d4e-aefa-b8887cd008a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:52.893341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd2a52-f1ac-4d4e-aefa-b8887cd008a1 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:52.899949 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76d4b5-faa3-4582-953e-88b2ec0c78dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb45ded9f75e4fa2973b564f83bdfaf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:52.900689 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76d4b5-faa3-4582-953e-88b2ec0c78dc service swift] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:52.901218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76d4b5-faa3-4582-953e-88b2ec0c78dc service swift] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:52.901870 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76d4b5-faa3-4582-953e-88b2ec0c78dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:52.912307 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76d4b5-faa3-4582-953e-88b2ec0c78dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:52.915351 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76d4b5-faa3-4582-953e-88b2ec0c78dc service swift] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:52.926101 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1335/2683] 10.222.0.74 () {60 vars in 1384 bytes} [Wed Jul 27 16:43:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 16:43:53.056393 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e859b-1bea-4e78-95a6-afd5e8415e4e None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:53.058131 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-a14e859b-1bea-4e78-95a6-afd5e8415e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:53.067344 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76d4b5-faa3-4582-953e-88b2ec0c78dc service swift] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:53.071296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-3e76d4b5-faa3-4582-953e-88b2ec0c78dc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:53.082922 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e859b-1bea-4e78-95a6-afd5e8415e4e None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:53.097040 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb129a5-1789-486b-81dc-0eb96725e488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:53.097962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb129a5-1789-486b-81dc-0eb96725e488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:53.098413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb129a5-1789-486b-81dc-0eb96725e488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:53.099107 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb129a5-1789-486b-81dc-0eb96725e488 admin admin] PATH_INFO: `/v3/projects/0488356968324807b46f2090f348d818` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:53.100685 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb129a5-1789-486b-81dc-0eb96725e488 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0488356968324807b46f2090f348d818)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:53.103078 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb129a5-1789-486b-81dc-0eb96725e488 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:53.137950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76d4b5-faa3-4582-953e-88b2ec0c78dc service swift] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:53.149561 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76496af9-66df-483a-a4e6-5f848f3c9712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:53.151099 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76496af9-66df-483a-a4e6-5f848f3c9712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:53.151672 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76496af9-66df-483a-a4e6-5f848f3c9712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:53.152534 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76496af9-66df-483a-a4e6-5f848f3c9712 admin admin] PATH_INFO: `/v3/projects/8c2d335bc9f6477d8909c3534fb3fbe9` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:53.158238 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76496af9-66df-483a-a4e6-5f848f3c9712 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c2d335bc9f6477d8909c3534fb3fbe9)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:53.161360 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76496af9-66df-483a-a4e6-5f848f3c9712 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:53.238906 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-aeb129a5-1789-486b-81dc-0eb96725e488 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0488356968324807b46f2090f348d818', 'request_id': 'req-aeb129a5-1789-486b-81dc-0eb96725e488'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:53.240960 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-76496af9-66df-483a-a4e6-5f848f3c9712 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c2d335bc9f6477d8909c3534fb3fbe9', 'request_id': 'req-76496af9-66df-483a-a4e6-5f848f3c9712'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:53.249413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-aeb129a5-1789-486b-81dc-0eb96725e488 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0488356968324807b46f2090f348d818', 'request_id': 'req-aeb129a5-1789-486b-81dc-0eb96725e488'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:53.251831 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:53.264232 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-aeb129a5-1789-486b-81dc-0eb96725e488 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0488356968324807b46f2090f348d818', 'request_id': 'req-aeb129a5-1789-486b-81dc-0eb96725e488'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:53.264744 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-76496af9-66df-483a-a4e6-5f848f3c9712 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c2d335bc9f6477d8909c3534fb3fbe9', 'request_id': 'req-76496af9-66df-483a-a4e6-5f848f3c9712'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:53.277448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-76496af9-66df-483a-a4e6-5f848f3c9712 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c2d335bc9f6477d8909c3534fb3fbe9', 'request_id': 'req-76496af9-66df-483a-a4e6-5f848f3c9712'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:53.286227 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-aeb129a5-1789-486b-81dc-0eb96725e488 admin admin] The token cache is being invalidate because project 0488356968324807b46f2090f348d818 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:43:53.286785 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-aeb129a5-1789-486b-81dc-0eb96725e488 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:53.287695 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-aeb129a5-1789-486b-81dc-0eb96725e488 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:53.289719 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1349/2684] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:43:53 2022] DELETE /identity/v3/projects/0488356968324807b46f2090f348d818 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:53.294211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb129a5-1789-486b-81dc-0eb96725e488 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:53.295925 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-aeb129a5-1789-486b-81dc-0eb96725e488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:53.318887 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-76496af9-66df-483a-a4e6-5f848f3c9712 admin admin] The token cache is being invalidate because project 8c2d335bc9f6477d8909c3534fb3fbe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:43:53.319191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-76496af9-66df-483a-a4e6-5f848f3c9712 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:53.320402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-76496af9-66df-483a-a4e6-5f848f3c9712 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:53.324650 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1336/2685] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:43:53 2022] DELETE /identity/v3/projects/8c2d335bc9f6477d8909c3534fb3fbe9 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:53.336020 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76496af9-66df-483a-a4e6-5f848f3c9712 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:53.336020 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-76496af9-66df-483a-a4e6-5f848f3c9712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:53.425387 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb129a5-1789-486b-81dc-0eb96725e488 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:53.429803 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:53.502558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76496af9-66df-483a-a4e6-5f848f3c9712 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:53.511542 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fec5a-25bb-49c5-9b89-34504712148e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:53.512908 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fec5a-25bb-49c5-9b89-34504712148e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:53.513353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fec5a-25bb-49c5-9b89-34504712148e admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:53.513755 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fec5a-25bb-49c5-9b89-34504712148e admin admin] PATH_INFO: `/v3/users/fe5c9f11b6ba469183cc57d37874ece1` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:53.515175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774fec5a-25bb-49c5-9b89-34504712148e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe5c9f11b6ba469183cc57d37874ece1)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:53.517566 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774fec5a-25bb-49c5-9b89-34504712148e admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:53.542043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638b060-2d9a-4834-9d5e-421b8cf90de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:53.542962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638b060-2d9a-4834-9d5e-421b8cf90de9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:53.543949 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638b060-2d9a-4834-9d5e-421b8cf90de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:53.545352 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638b060-2d9a-4834-9d5e-421b8cf90de9 admin admin] PATH_INFO: `/v3/users/c257bb3b3abd430698dc4bfadda7cd21` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:53.547873 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638b060-2d9a-4834-9d5e-421b8cf90de9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c257bb3b3abd430698dc4bfadda7cd21)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:53.551767 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638b060-2d9a-4834-9d5e-421b8cf90de9 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:53.586592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-774fec5a-25bb-49c5-9b89-34504712148e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe5c9f11b6ba469183cc57d37874ece1'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:53.598615 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-774fec5a-25bb-49c5-9b89-34504712148e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe5c9f11b6ba469183cc57d37874ece1'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:53.611866 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-774fec5a-25bb-49c5-9b89-34504712148e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe5c9f11b6ba469183cc57d37874ece1'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:53.621409 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-2638b060-2d9a-4834-9d5e-421b8cf90de9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c257bb3b3abd430698dc4bfadda7cd21'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:53.629380 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-774fec5a-25bb-49c5-9b89-34504712148e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe5c9f11b6ba469183cc57d37874ece1'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:53.631711 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-774fec5a-25bb-49c5-9b89-34504712148e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe5c9f11b6ba469183cc57d37874ece1'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:53.638006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-2638b060-2d9a-4834-9d5e-421b8cf90de9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c257bb3b3abd430698dc4bfadda7cd21'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:53.641140 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1337/2686] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:43:53 2022] DELETE /identity/v3/users/fe5c9f11b6ba469183cc57d37874ece1 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:53.654619 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-2638b060-2d9a-4834-9d5e-421b8cf90de9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c257bb3b3abd430698dc4bfadda7cd21'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:53.662635 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-2638b060-2d9a-4834-9d5e-421b8cf90de9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c257bb3b3abd430698dc4bfadda7cd21'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:53.663776 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-2638b060-2d9a-4834-9d5e-421b8cf90de9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c257bb3b3abd430698dc4bfadda7cd21'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:53.674516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1350/2687] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:43:53 2022] DELETE /identity/v3/users/c257bb3b3abd430698dc4bfadda7cd21 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:53.974858 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fec5a-25bb-49c5-9b89-34504712148e admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:53.976811 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-774fec5a-25bb-49c5-9b89-34504712148e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:54.069696 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fec5a-25bb-49c5-9b89-34504712148e admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:54.129852 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054dc025-c798-48eb-8696-13b8c85feed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:54.130169 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054dc025-c798-48eb-8696-13b8c85feed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:54.130453 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054dc025-c798-48eb-8696-13b8c85feed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:54.130825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054dc025-c798-48eb-8696-13b8c85feed2 admin admin] PATH_INFO: `/v3/projects/a15ec38be3aa4c29991b007b0e5f7566` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:54.132091 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054dc025-c798-48eb-8696-13b8c85feed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a15ec38be3aa4c29991b007b0e5f7566)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:54.134061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054dc025-c798-48eb-8696-13b8c85feed2 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:54.146145 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:54.194136 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-054dc025-c798-48eb-8696-13b8c85feed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a15ec38be3aa4c29991b007b0e5f7566', 'request_id': 'req-054dc025-c798-48eb-8696-13b8c85feed2'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:54.200071 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-054dc025-c798-48eb-8696-13b8c85feed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a15ec38be3aa4c29991b007b0e5f7566', 'request_id': 'req-054dc025-c798-48eb-8696-13b8c85feed2'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:54.207733 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-054dc025-c798-48eb-8696-13b8c85feed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a15ec38be3aa4c29991b007b0e5f7566', 'request_id': 'req-054dc025-c798-48eb-8696-13b8c85feed2'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:54.221067 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-054dc025-c798-48eb-8696-13b8c85feed2 admin admin] The token cache is being invalidate because project a15ec38be3aa4c29991b007b0e5f7566 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:43:54.221365 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-054dc025-c798-48eb-8696-13b8c85feed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:54.222011 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-054dc025-c798-48eb-8696-13b8c85feed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:54.223542 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1338/2688] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:43:53 2022] DELETE /identity/v3/projects/a15ec38be3aa4c29991b007b0e5f7566 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:54.228753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178b41a-d5d9-440e-8b71-ed6aebd19149 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:54.229228 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178b41a-d5d9-440e-8b71-ed6aebd19149 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:54.229341 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178b41a-d5d9-440e-8b71-ed6aebd19149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:54.250125 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-c178b41a-d5d9-440e-8b71-ed6aebd19149 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:54.251343 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-c178b41a-d5d9-440e-8b71-ed6aebd19149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:54.257128 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:54.319506 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1351/2689] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:54.328193 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d47b3-6fdd-421c-b289-6eb2e0ff0683 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:54.328426 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d47b3-6fdd-421c-b289-6eb2e0ff0683 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:54.328671 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d47b3-6fdd-421c-b289-6eb2e0ff0683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:54.352499 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-365d47b3-6fdd-421c-b289-6eb2e0ff0683 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:54.353812 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-365d47b3-6fdd-421c-b289-6eb2e0ff0683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:54.362565 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1339/2690] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:54.366749 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178b41a-d5d9-440e-8b71-ed6aebd19149 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:54.377950 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178b41a-d5d9-440e-8b71-ed6aebd19149 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:54.383275 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784a65c-de6b-44d3-965a-ac96dc450666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:54.383905 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784a65c-de6b-44d3-965a-ac96dc450666 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:54.384186 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784a65c-de6b-44d3-965a-ac96dc450666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:54.384480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784a65c-de6b-44d3-965a-ac96dc450666 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:54.385248 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784a65c-de6b-44d3-965a-ac96dc450666 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:43:54.385559 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784a65c-de6b-44d3-965a-ac96dc450666 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:54.387729 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784a65c-de6b-44d3-965a-ac96dc450666 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:54.405692 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1352/2691] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:43:54.407233 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d47b3-6fdd-421c-b289-6eb2e0ff0683 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:54.417713 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d47b3-6fdd-421c-b289-6eb2e0ff0683 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:54.423465 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9803f90-1195-4920-a46a-0e24abfa9408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:54.424101 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9803f90-1195-4920-a46a-0e24abfa9408 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:54.424387 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9803f90-1195-4920-a46a-0e24abfa9408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:54.424667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9803f90-1195-4920-a46a-0e24abfa9408 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:54.425514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9803f90-1195-4920-a46a-0e24abfa9408 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:54.427463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9803f90-1195-4920-a46a-0e24abfa9408 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:54.461568 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1340/2692] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:43:54 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:54.466441 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784a65c-de6b-44d3-965a-ac96dc450666 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:54.479859 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9803f90-1195-4920-a46a-0e24abfa9408 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:54.482068 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-f9803f90-1195-4920-a46a-0e24abfa9408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:54.487537 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784a65c-de6b-44d3-965a-ac96dc450666 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:54.573973 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aee166-926b-40fb-a255-36c2e536dfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:54.575354 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aee166-926b-40fb-a255-36c2e536dfb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:54.575744 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aee166-926b-40fb-a255-36c2e536dfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:54.576117 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aee166-926b-40fb-a255-36c2e536dfb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:54.576923 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aee166-926b-40fb-a255-36c2e536dfb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:54.579319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aee166-926b-40fb-a255-36c2e536dfb9 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:54.590801 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9803f90-1195-4920-a46a-0e24abfa9408 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:54.598626 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b597d1-4d61-41fa-9471-7483698b1bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:54.599329 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b597d1-4d61-41fa-9471-7483698b1bf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:54.599650 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b597d1-4d61-41fa-9471-7483698b1bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:54.599918 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b597d1-4d61-41fa-9471-7483698b1bf5 admin admin] PATH_INFO: `/v3/projects/3dfdec6d4cdd4012920c01dc1907c4cc` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:54.601698 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b597d1-4d61-41fa-9471-7483698b1bf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dfdec6d4cdd4012920c01dc1907c4cc)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:54.603775 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b597d1-4d61-41fa-9471-7483698b1bf5 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:54.616094 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1353/2693] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:43:54 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:43:54.619975 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aee166-926b-40fb-a255-36c2e536dfb9 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:54.639475 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aee166-926b-40fb-a255-36c2e536dfb9 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:54.651280 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9aa87a-92a8-415d-a151-a065f068b160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:54.653214 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9aa87a-92a8-415d-a151-a065f068b160 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:54.653500 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9aa87a-92a8-415d-a151-a065f068b160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:54.653768 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9aa87a-92a8-415d-a151-a065f068b160 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:54.654488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9aa87a-92a8-415d-a151-a065f068b160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:54.658000 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9aa87a-92a8-415d-a151-a065f068b160 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:54.669179 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-82b597d1-4d61-41fa-9471-7483698b1bf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dfdec6d4cdd4012920c01dc1907c4cc', 'request_id': 'req-82b597d1-4d61-41fa-9471-7483698b1bf5'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:54.677606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-82b597d1-4d61-41fa-9471-7483698b1bf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dfdec6d4cdd4012920c01dc1907c4cc', 'request_id': 'req-82b597d1-4d61-41fa-9471-7483698b1bf5'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:54.680995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1354/2694] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:54.685158 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9aa87a-92a8-415d-a151-a065f068b160 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:54.689037 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-82b597d1-4d61-41fa-9471-7483698b1bf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dfdec6d4cdd4012920c01dc1907c4cc', 'request_id': 'req-82b597d1-4d61-41fa-9471-7483698b1bf5'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:54.696512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9aa87a-92a8-415d-a151-a065f068b160 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:54.709154 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-82b597d1-4d61-41fa-9471-7483698b1bf5 admin admin] The token cache is being invalidate because project 3dfdec6d4cdd4012920c01dc1907c4cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:43:54.709589 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-82b597d1-4d61-41fa-9471-7483698b1bf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:54.710378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-82b597d1-4d61-41fa-9471-7483698b1bf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:54.715885 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1341/2695] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:43:54 2022] DELETE /identity/v3/projects/3dfdec6d4cdd4012920c01dc1907c4cc => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:43:54.717453 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33afc35a-98c3-4431-8cfb-502ffb7386c8 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:54.717891 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33afc35a-98c3-4431-8cfb-502ffb7386c8 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:54.718272 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33afc35a-98c3-4431-8cfb-502ffb7386c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:54.729032 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:54.748839 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-33afc35a-98c3-4431-8cfb-502ffb7386c8 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:54.750499 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-33afc35a-98c3-4431-8cfb-502ffb7386c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:54.792482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b95c7-079d-4d98-8e6d-492d98085ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:54.793227 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b95c7-079d-4d98-8e6d-492d98085ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:54.794037 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b95c7-079d-4d98-8e6d-492d98085ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:54.794318 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b95c7-079d-4d98-8e6d-492d98085ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:54.795021 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b95c7-079d-4d98-8e6d-492d98085ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:54.797075 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b95c7-079d-4d98-8e6d-492d98085ed0 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:54.816868 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1355/2696] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:43:54.817985 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b95c7-079d-4d98-8e6d-492d98085ed0 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:54.819754 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-bb8b95c7-079d-4d98-8e6d-492d98085ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:54.859093 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b95c7-079d-4d98-8e6d-492d98085ed0 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:54.873577 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d2764-0f87-4942-860a-1da802e03171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:54.875947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d2764-0f87-4942-860a-1da802e03171 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:54.877143 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d2764-0f87-4942-860a-1da802e03171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:54.878713 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d2764-0f87-4942-860a-1da802e03171 admin admin] PATH_INFO: `/v3/projects/8ef15b981e8d4885941afecc5cc74d1a/users/8b5a532003384e5180c73124c7b62077/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:54.886579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1342/2697] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:54.895744 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace889cc-41eb-495d-b830-f0f19e6de0c0 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:54.896041 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace889cc-41eb-495d-b830-f0f19e6de0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:54.896323 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace889cc-41eb-495d-b830-f0f19e6de0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:54.899045 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:54.908875 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d2764-0f87-4942-860a-1da802e03171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ef15b981e8d4885941afecc5cc74d1a, user_id=8b5a532003384e5180c73124c7b62077, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:54.912046 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d2764-0f87-4942-860a-1da802e03171 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:54.926445 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-ace889cc-41eb-495d-b830-f0f19e6de0c0 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:54.928096 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ace889cc-41eb-495d-b830-f0f19e6de0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:54.937292 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1356/2698] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:54 2022] PUT /identity/v3/projects/8ef15b981e8d4885941afecc5cc74d1a/users/8b5a532003384e5180c73124c7b62077/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:54.942155 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfd7e3-f7b7-411a-a745-339dd5f4b065 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:54.943720 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfd7e3-f7b7-411a-a745-339dd5f4b065 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:54.944112 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfd7e3-f7b7-411a-a745-339dd5f4b065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:54.991658 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-63bfd7e3-f7b7-411a-a745-339dd5f4b065 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:54.996423 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-63bfd7e3-f7b7-411a-a745-339dd5f4b065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:55.003861 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1343/2699] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:55.009093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace889cc-41eb-495d-b830-f0f19e6de0c0 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:55.019081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1357/2700] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:43:55.026906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace889cc-41eb-495d-b830-f0f19e6de0c0 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:55.029664 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfd7e3-f7b7-411a-a745-339dd5f4b065 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:55.031498 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-63bfd7e3-f7b7-411a-a745-339dd5f4b065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:55.036208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c51a5-8b9e-46c3-a2d9-814e30d313a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:55.037161 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c51a5-8b9e-46c3-a2d9-814e30d313a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:55.037723 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c51a5-8b9e-46c3-a2d9-814e30d313a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:55.038083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c51a5-8b9e-46c3-a2d9-814e30d313a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:55.039085 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c51a5-8b9e-46c3-a2d9-814e30d313a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:43:55.040413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c51a5-8b9e-46c3-a2d9-814e30d313a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:55.044582 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c51a5-8b9e-46c3-a2d9-814e30d313a4 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:55.059417 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1344/2701] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:43:55.064493 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c51a5-8b9e-46c3-a2d9-814e30d313a4 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:55.078314 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c51a5-8b9e-46c3-a2d9-814e30d313a4 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:55.084356 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af14b23-4031-4635-a9fb-babdff407759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:55.085353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af14b23-4031-4635-a9fb-babdff407759 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:55.085781 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af14b23-4031-4635-a9fb-babdff407759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:55.086035 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af14b23-4031-4635-a9fb-babdff407759 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:55.087737 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af14b23-4031-4635-a9fb-babdff407759 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:55.091716 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af14b23-4031-4635-a9fb-babdff407759 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:55.138973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1345/2702] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:43:55 2022] POST /identity/v3/projects => generated 381 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:55.140622 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af14b23-4031-4635-a9fb-babdff407759 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:55.141326 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfd7e3-f7b7-411a-a745-339dd5f4b065 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:55.211496 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5cf581-c0c9-47b5-ba5f-d0ce10bf0b16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:55.212237 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cf581-c0c9-47b5-ba5f-d0ce10bf0b16 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:55.212619 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cf581-c0c9-47b5-ba5f-d0ce10bf0b16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:55.213026 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cf581-c0c9-47b5-ba5f-d0ce10bf0b16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:55.251014 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af14b23-4031-4635-a9fb-babdff407759 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:55.256309 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84898f2f-599e-4839-8d21-78c56315b722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:55.258056 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84898f2f-599e-4839-8d21-78c56315b722 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:55.258056 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84898f2f-599e-4839-8d21-78c56315b722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:55.258276 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84898f2f-599e-4839-8d21-78c56315b722 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:55.258832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84898f2f-599e-4839-8d21-78c56315b722 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:55.260929 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84898f2f-599e-4839-8d21-78c56315b722 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:55.282708 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:55.292539 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1346/2703] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:43:55 2022] POST /identity/v3/users => generated 291 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:55.297678 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84898f2f-599e-4839-8d21-78c56315b722 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:55.308170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5cf581-c0c9-47b5-ba5f-d0ce10bf0b16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:55.309007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84898f2f-599e-4839-8d21-78c56315b722 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:55.309738 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5cf581-c0c9-47b5-ba5f-d0ce10bf0b16 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:55.314080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b43d8-0946-46a5-a2bc-b7529568bde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:55.314951 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b43d8-0946-46a5-a2bc-b7529568bde5 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:55.315248 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b43d8-0946-46a5-a2bc-b7529568bde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:55.315756 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b43d8-0946-46a5-a2bc-b7529568bde5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:55.317182 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b43d8-0946-46a5-a2bc-b7529568bde5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:55.319331 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b43d8-0946-46a5-a2bc-b7529568bde5 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:55.325048 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1358/2704] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:43:55.339588 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1347/2705] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:43:55.344087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5cf581-c0c9-47b5-ba5f-d0ce10bf0b16 service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:55.358529 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5cf581-c0c9-47b5-ba5f-d0ce10bf0b16 service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:55.364001 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3dbf6-9be7-42b2-9957-6c702bd92902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:55.364749 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3dbf6-9be7-42b2-9957-6c702bd92902 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:55.365345 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3dbf6-9be7-42b2-9957-6c702bd92902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:55.365880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3dbf6-9be7-42b2-9957-6c702bd92902 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:55.366460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d3dbf6-9be7-42b2-9957-6c702bd92902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:55.368614 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d3dbf6-9be7-42b2-9957-6c702bd92902 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:55.383938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1359/2706] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:43:55.388666 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b43d8-0946-46a5-a2bc-b7529568bde5 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:55.406964 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b43d8-0946-46a5-a2bc-b7529568bde5 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:55.413975 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33bad1-6dfd-4373-a26b-68549f54f483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:55.415183 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33bad1-6dfd-4373-a26b-68549f54f483 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:55.416111 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33bad1-6dfd-4373-a26b-68549f54f483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:55.416601 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33bad1-6dfd-4373-a26b-68549f54f483 admin admin] PATH_INFO: `/v3/projects/9f92f5257bcc4e048eeadc7ae4981123/users/e8e5f63df8ac4f16846fe18308aeb0a3/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:55.438952 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33bad1-6dfd-4373-a26b-68549f54f483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f92f5257bcc4e048eeadc7ae4981123, user_id=e8e5f63df8ac4f16846fe18308aeb0a3, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:55.448748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33bad1-6dfd-4373-a26b-68549f54f483 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:55.471090 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1348/2707] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:55 2022] PUT /identity/v3/projects/9f92f5257bcc4e048eeadc7ae4981123/users/e8e5f63df8ac4f16846fe18308aeb0a3/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:55.482608 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc29048-1d15-4d56-bc92-00149166a2ab None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:55.482980 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc29048-1d15-4d56-bc92-00149166a2ab None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:55.483384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc29048-1d15-4d56-bc92-00149166a2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:55.520433 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-bcc29048-1d15-4d56-bc92-00149166a2ab None None] MFA Rules not processed for user `e8e5f63df8ac4f16846fe18308aeb0a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:55.522576 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-bcc29048-1d15-4d56-bc92-00149166a2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:55.567215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:55.633008 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1360/2708] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:55.638360 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33bad1-6dfd-4373-a26b-68549f54f483 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:55.706662 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33bad1-6dfd-4373-a26b-68549f54f483 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:55.711620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e96f9-5b3b-4034-8eef-b0f1cc7296d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:55.712313 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e96f9-5b3b-4034-8eef-b0f1cc7296d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:55.712653 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e96f9-5b3b-4034-8eef-b0f1cc7296d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:55.713019 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e96f9-5b3b-4034-8eef-b0f1cc7296d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:55.713792 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e96f9-5b3b-4034-8eef-b0f1cc7296d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:55.715782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e96f9-5b3b-4034-8eef-b0f1cc7296d3 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:55.759888 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1349/2709] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:43:55 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:43:55.761122 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc29048-1d15-4d56-bc92-00149166a2ab None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:55.775811 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc29048-1d15-4d56-bc92-00149166a2ab None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:55.844379 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d6328-5e07-4f6f-9e7b-3e3ea002891b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:55.845967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d6328-5e07-4f6f-9e7b-3e3ea002891b admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:55.846535 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d6328-5e07-4f6f-9e7b-3e3ea002891b admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:55.847018 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d6328-5e07-4f6f-9e7b-3e3ea002891b admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:55.848298 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d6328-5e07-4f6f-9e7b-3e3ea002891b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:55.852102 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d6328-5e07-4f6f-9e7b-3e3ea002891b admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:55.883957 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1361/2710] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:43:55 2022] POST /identity/v3/users => generated 297 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:55.888127 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e96f9-5b3b-4034-8eef-b0f1cc7296d3 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:55.899258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e96f9-5b3b-4034-8eef-b0f1cc7296d3 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:55.905066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4fff1-c227-43d1-8b04-0da0e777e142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:55.905681 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4fff1-c227-43d1-8b04-0da0e777e142 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:55.906352 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4fff1-c227-43d1-8b04-0da0e777e142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:55.906352 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4fff1-c227-43d1-8b04-0da0e777e142 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:55.907097 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da4fff1-c227-43d1-8b04-0da0e777e142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:55.908717 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da4fff1-c227-43d1-8b04-0da0e777e142 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:55.920927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1350/2711] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:55.925455 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d6328-5e07-4f6f-9e7b-3e3ea002891b admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:55.936505 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d6328-5e07-4f6f-9e7b-3e3ea002891b admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:55.941511 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5c91e-ef8d-495e-82b1-1880da6856a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:55.942190 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5c91e-ef8d-495e-82b1-1880da6856a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:55.942477 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5c91e-ef8d-495e-82b1-1880da6856a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:55.942755 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5c91e-ef8d-495e-82b1-1880da6856a3 admin admin] PATH_INFO: `/v3/projects/2bedc51874a14011bc47768b10c42752/users/bd93e6ae088b4afaa317bd68a281463e/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:55.963383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5c91e-ef8d-495e-82b1-1880da6856a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bedc51874a14011bc47768b10c42752, user_id=bd93e6ae088b4afaa317bd68a281463e, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:55.967166 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5c91e-ef8d-495e-82b1-1880da6856a3 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:55.991541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1362/2712] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:55 2022] PUT /identity/v3/projects/2bedc51874a14011bc47768b10c42752/users/bd93e6ae088b4afaa317bd68a281463e/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:43:55.995579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4fff1-c227-43d1-8b04-0da0e777e142 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:56.010561 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4fff1-c227-43d1-8b04-0da0e777e142 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:56.135103 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c312ea-1c77-46ec-b3a3-7a8dc7d03366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:56.137646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c312ea-1c77-46ec-b3a3-7a8dc7d03366 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:56.138613 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c312ea-1c77-46ec-b3a3-7a8dc7d03366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:56.140026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c312ea-1c77-46ec-b3a3-7a8dc7d03366 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:56.143400 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c312ea-1c77-46ec-b3a3-7a8dc7d03366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:56.145393 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c312ea-1c77-46ec-b3a3-7a8dc7d03366 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:56.180139 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d10feb-0f88-4bc8-b373-514c9125a712 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:56.180567 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d10feb-0f88-4bc8-b373-514c9125a712 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:56.180975 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d10feb-0f88-4bc8-b373-514c9125a712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:56.205856 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1351/2713] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:43:56.211521 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c312ea-1c77-46ec-b3a3-7a8dc7d03366 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:56.229853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-79d10feb-0f88-4bc8-b373-514c9125a712 None None] MFA Rules not processed for user `8b5a532003384e5180c73124c7b62077`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:56.231238 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-79d10feb-0f88-4bc8-b373-514c9125a712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:56.235146 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c312ea-1c77-46ec-b3a3-7a8dc7d03366 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:56.252645 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3277d5-3e31-4ae9-895b-d33af6ee9053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:56.253380 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3277d5-3e31-4ae9-895b-d33af6ee9053 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:56.253667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3277d5-3e31-4ae9-895b-d33af6ee9053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:56.253947 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3277d5-3e31-4ae9-895b-d33af6ee9053 admin admin] PATH_INFO: `/v3/projects/2bedc51874a14011bc47768b10c42752/users/bd93e6ae088b4afaa317bd68a281463e/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:56.256579 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3277d5-3e31-4ae9-895b-d33af6ee9053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bedc51874a14011bc47768b10c42752, user_id=bd93e6ae088b4afaa317bd68a281463e, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:56.260930 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3277d5-3e31-4ae9-895b-d33af6ee9053 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:56.300444 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1352/2714] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:56 2022] PUT /identity/v3/projects/2bedc51874a14011bc47768b10c42752/users/bd93e6ae088b4afaa317bd68a281463e/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:56.302480 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a66e84-d68f-469b-8510-09a675062f25 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:56.303313 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a66e84-d68f-469b-8510-09a675062f25 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:56.303606 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a66e84-d68f-469b-8510-09a675062f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:56.358880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-05a66e84-d68f-469b-8510-09a675062f25 None None] MFA Rules not processed for user `bd93e6ae088b4afaa317bd68a281463e`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:56.360175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-05a66e84-d68f-469b-8510-09a675062f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:56.372574 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:56.373137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1363/2715] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:43:56.375367 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d10feb-0f88-4bc8-b373-514c9125a712 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:56.402337 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d10feb-0f88-4bc8-b373-514c9125a712 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:56.437789 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:56.480248 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1353/2716] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:56 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:43:56.489006 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a66e84-d68f-469b-8510-09a675062f25 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:56.491535 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-05a66e84-d68f-469b-8510-09a675062f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:56.503993 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8dac3b-9303-4b08-b71c-b3f630037627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:56.504850 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dac3b-9303-4b08-b71c-b3f630037627 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:56.506263 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dac3b-9303-4b08-b71c-b3f630037627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:56.506836 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dac3b-9303-4b08-b71c-b3f630037627 admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:56.507655 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8dac3b-9303-4b08-b71c-b3f630037627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:56.510421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8dac3b-9303-4b08-b71c-b3f630037627 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:56.553856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1364/2717] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:43:56 2022] POST /identity/v3/projects => generated 371 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:56.557721 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8dac3b-9303-4b08-b71c-b3f630037627 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:56.618066 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a66e84-d68f-469b-8510-09a675062f25 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:56.692986 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8dac3b-9303-4b08-b71c-b3f630037627 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:56.708915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8df4d5-648d-46f2-8223-71028c23007f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:56.708915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8df4d5-648d-46f2-8223-71028c23007f admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:56.708915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8df4d5-648d-46f2-8223-71028c23007f admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:56.708915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8df4d5-648d-46f2-8223-71028c23007f admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:56.709380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8df4d5-648d-46f2-8223-71028c23007f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:56.709380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8df4d5-648d-46f2-8223-71028c23007f admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:56.753099 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249242d-a3d3-442b-aa7d-20c8826b420b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:56.754108 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249242d-a3d3-442b-aa7d-20c8826b420b service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:56.754493 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249242d-a3d3-442b-aa7d-20c8826b420b service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:56.755124 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249242d-a3d3-442b-aa7d-20c8826b420b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:56.763990 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1365/2718] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:43:56 2022] POST /identity/v3/users => generated 292 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:43:56.768171 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8df4d5-648d-46f2-8223-71028c23007f admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:56.784678 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8df4d5-648d-46f2-8223-71028c23007f admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:56.795137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b511d6-9c85-4243-892d-7d6efcb46c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:56.796391 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b511d6-9c85-4243-892d-7d6efcb46c76 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:56.797000 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b511d6-9c85-4243-892d-7d6efcb46c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:56.797516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b511d6-9c85-4243-892d-7d6efcb46c76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:56.798866 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b511d6-9c85-4243-892d-7d6efcb46c76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:56.803527 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b511d6-9c85-4243-892d-7d6efcb46c76 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:56.824263 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1366/2719] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:56.831705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b511d6-9c85-4243-892d-7d6efcb46c76 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:56.843625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249242d-a3d3-442b-aa7d-20c8826b420b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:56.844915 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249242d-a3d3-442b-aa7d-20c8826b420b service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:56.852553 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b511d6-9c85-4243-892d-7d6efcb46c76 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:56.856418 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1354/2720] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:56 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:43:56.858588 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65ea29-0d6f-4882-8bde-b0bdd478dd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:56.859580 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65ea29-0d6f-4882-8bde-b0bdd478dd5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:56.860427 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65ea29-0d6f-4882-8bde-b0bdd478dd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:56.860976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65ea29-0d6f-4882-8bde-b0bdd478dd5c admin admin] PATH_INFO: `/v3/projects/7e8ea5b970244614987d988e0f9673b7/users/5923756dc65442e692e33f52a4bc1f49/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:56.884615 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf65ea29-0d6f-4882-8bde-b0bdd478dd5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e8ea5b970244614987d988e0f9673b7, user_id=5923756dc65442e692e33f52a4bc1f49, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:56.895006 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf65ea29-0d6f-4882-8bde-b0bdd478dd5c admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:56.911469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1367/2721] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:56 2022] PUT /identity/v3/projects/7e8ea5b970244614987d988e0f9673b7/users/5923756dc65442e692e33f52a4bc1f49/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:56.915972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249242d-a3d3-442b-aa7d-20c8826b420b service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:56.932602 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249242d-a3d3-442b-aa7d-20c8826b420b service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:56.944230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65ea29-0d6f-4882-8bde-b0bdd478dd5c admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:56.948007 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:57.020240 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65ea29-0d6f-4882-8bde-b0bdd478dd5c admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:57.020674 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f178ba2-0216-43f0-90fb-016216d2c966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:57.021195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f178ba2-0216-43f0-90fb-016216d2c966 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:57.021539 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f178ba2-0216-43f0-90fb-016216d2c966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:57.021827 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f178ba2-0216-43f0-90fb-016216d2c966 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:57.022530 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f178ba2-0216-43f0-90fb-016216d2c966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:57.024465 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f178ba2-0216-43f0-90fb-016216d2c966 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:57.027761 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d622fb-9f12-4423-88f0-1e40119ed20e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:57.028877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d622fb-9f12-4423-88f0-1e40119ed20e service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:57.029382 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d622fb-9f12-4423-88f0-1e40119ed20e service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:57.029848 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d622fb-9f12-4423-88f0-1e40119ed20e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:57.042888 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1355/2722] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:43:57.046324 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f178ba2-0216-43f0-90fb-016216d2c966 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:57.050307 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:57.060389 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f178ba2-0216-43f0-90fb-016216d2c966 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:57.081657 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9493a-2fb5-4099-b6ed-04a7ccfb80c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:57.082382 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9493a-2fb5-4099-b6ed-04a7ccfb80c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:57.082862 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9493a-2fb5-4099-b6ed-04a7ccfb80c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:57.082984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9493a-2fb5-4099-b6ed-04a7ccfb80c9 admin admin] PATH_INFO: `/v3/projects/7e8ea5b970244614987d988e0f9673b7/users/5923756dc65442e692e33f52a4bc1f49/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:57.085341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9493a-2fb5-4099-b6ed-04a7ccfb80c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e8ea5b970244614987d988e0f9673b7, user_id=5923756dc65442e692e33f52a4bc1f49, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:57.087958 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9493a-2fb5-4099-b6ed-04a7ccfb80c9 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:57.100648 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1356/2723] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:43:57 2022] PUT /identity/v3/projects/7e8ea5b970244614987d988e0f9673b7/users/5923756dc65442e692e33f52a4bc1f49/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:57.128948 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d622fb-9f12-4423-88f0-1e40119ed20e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:57.130457 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d622fb-9f12-4423-88f0-1e40119ed20e service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:57.142944 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1368/2724] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:43:57.520552 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9493a-2fb5-4099-b6ed-04a7ccfb80c9 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:57.606302 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9493a-2fb5-4099-b6ed-04a7ccfb80c9 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:57.613796 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1fed3b-faab-4ea6-9c7a-b0f06d37ad30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:57.614823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1fed3b-faab-4ea6-9c7a-b0f06d37ad30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:57.615538 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1fed3b-faab-4ea6-9c7a-b0f06d37ad30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:57.615973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1fed3b-faab-4ea6-9c7a-b0f06d37ad30 admin admin] PATH_INFO: `/v3/users/e8e5f63df8ac4f16846fe18308aeb0a3` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:57.618195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1fed3b-faab-4ea6-9c7a-b0f06d37ad30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8e5f63df8ac4f16846fe18308aeb0a3)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:57.620418 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1fed3b-faab-4ea6-9c7a-b0f06d37ad30 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:57.756063 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8e1fed3b-faab-4ea6-9c7a-b0f06d37ad30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8e5f63df8ac4f16846fe18308aeb0a3'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:57.779015 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8e1fed3b-faab-4ea6-9c7a-b0f06d37ad30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8e5f63df8ac4f16846fe18308aeb0a3'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:57.796061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8e1fed3b-faab-4ea6-9c7a-b0f06d37ad30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8e5f63df8ac4f16846fe18308aeb0a3'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:57.843828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8e1fed3b-faab-4ea6-9c7a-b0f06d37ad30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8e5f63df8ac4f16846fe18308aeb0a3'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:57.845152 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8e1fed3b-faab-4ea6-9c7a-b0f06d37ad30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8e5f63df8ac4f16846fe18308aeb0a3'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:57.868069 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1357/2725] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:43:57 2022] DELETE /identity/v3/users/e8e5f63df8ac4f16846fe18308aeb0a3 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:57.870195 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201a979-a28e-4702-ab71-b952b97418f5 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:57.870704 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201a979-a28e-4702-ab71-b952b97418f5 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:57.870804 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201a979-a28e-4702-ab71-b952b97418f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:57.898801 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-7201a979-a28e-4702-ab71-b952b97418f5 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:57.900008 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7201a979-a28e-4702-ab71-b952b97418f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:58.055076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1369/2726] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:43:58.088218 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1fed3b-faab-4ea6-9c7a-b0f06d37ad30 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:58.096640 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-8e1fed3b-faab-4ea6-9c7a-b0f06d37ad30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:58.125166 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:58.156092 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201a979-a28e-4702-ab71-b952b97418f5 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:58.160138 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7201a979-a28e-4702-ab71-b952b97418f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:58.217223 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1fed3b-faab-4ea6-9c7a-b0f06d37ad30 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:58.223240 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f59707-fa28-4da2-b699-626d3b14f981 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:58.224935 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f59707-fa28-4da2-b699-626d3b14f981 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:58.224935 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f59707-fa28-4da2-b699-626d3b14f981 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:58.225788 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f59707-fa28-4da2-b699-626d3b14f981 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:58.240222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f59707-fa28-4da2-b699-626d3b14f981 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:58.241729 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f59707-fa28-4da2-b699-626d3b14f981 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:58.272082 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1358/2727] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:58.307055 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201a979-a28e-4702-ab71-b952b97418f5 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:58.319320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8da73-5eed-4afc-9fbf-12e894cbbf82 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:58.319621 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8da73-5eed-4afc-9fbf-12e894cbbf82 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:58.319895 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8da73-5eed-4afc-9fbf-12e894cbbf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:58.321404 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff70b17-734c-40bb-bcd9-9a8fdfcafe9e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f8e0f82a8541aead6cd8cc45f4a86b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:58.322043 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff70b17-734c-40bb-bcd9-9a8fdfcafe9e service placement] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:58.322328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff70b17-734c-40bb-bcd9-9a8fdfcafe9e service placement] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:58.322742 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff70b17-734c-40bb-bcd9-9a8fdfcafe9e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:58.325224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7ff70b17-734c-40bb-bcd9-9a8fdfcafe9e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:58.338268 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff70b17-734c-40bb-bcd9-9a8fdfcafe9e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:58.339878 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff70b17-734c-40bb-bcd9-9a8fdfcafe9e service placement] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:58.352581 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1370/2728] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:43:58.354934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-ece8da73-5eed-4afc-9fbf-12e894cbbf82 None None] MFA Rules not processed for user `5923756dc65442e692e33f52a4bc1f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:58.356333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ece8da73-5eed-4afc-9fbf-12e894cbbf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:58.445948 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1359/2729] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:43:58 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:43:58.458730 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff70b17-734c-40bb-bcd9-9a8fdfcafe9e service placement] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:58.462200 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7ff70b17-734c-40bb-bcd9-9a8fdfcafe9e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:58.529105 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:58.562090 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff70b17-734c-40bb-bcd9-9a8fdfcafe9e service placement] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:58.566052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824a4ea-4dc9-4a06-8012-d7e51dd5ce41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:58.566709 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824a4ea-4dc9-4a06-8012-d7e51dd5ce41 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:58.566958 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824a4ea-4dc9-4a06-8012-d7e51dd5ce41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:58.567230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824a4ea-4dc9-4a06-8012-d7e51dd5ce41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:58.569100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-8824a4ea-4dc9-4a06-8012-d7e51dd5ce41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:58.654743 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824a4ea-4dc9-4a06-8012-d7e51dd5ce41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:58.656539 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824a4ea-4dc9-4a06-8012-d7e51dd5ce41 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:58.672031 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1371/2730] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:43:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:43:59.079845 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8da73-5eed-4afc-9fbf-12e894cbbf82 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:59.082941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ece8da73-5eed-4afc-9fbf-12e894cbbf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:59.107658 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8da73-5eed-4afc-9fbf-12e894cbbf82 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:59.133662 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def1fae-30c8-486c-bd33-d86e29269028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:59.134763 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def1fae-30c8-486c-bd33-d86e29269028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:59.135859 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def1fae-30c8-486c-bd33-d86e29269028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:59.137209 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def1fae-30c8-486c-bd33-d86e29269028 admin admin] PATH_INFO: `/v3/projects/9f92f5257bcc4e048eeadc7ae4981123` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:59.139341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def1fae-30c8-486c-bd33-d86e29269028 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f92f5257bcc4e048eeadc7ae4981123)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:59.142026 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def1fae-30c8-486c-bd33-d86e29269028 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:59.259562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6def1fae-30c8-486c-bd33-d86e29269028 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f92f5257bcc4e048eeadc7ae4981123', 'request_id': 'req-6def1fae-30c8-486c-bd33-d86e29269028'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:59.270311 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6def1fae-30c8-486c-bd33-d86e29269028 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f92f5257bcc4e048eeadc7ae4981123', 'request_id': 'req-6def1fae-30c8-486c-bd33-d86e29269028'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:59.276616 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167acf4-8fd5-4fd6-b640-98674366a053 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:59.277268 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167acf4-8fd5-4fd6-b640-98674366a053 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:59.277268 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167acf4-8fd5-4fd6-b640-98674366a053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:59.280843 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6def1fae-30c8-486c-bd33-d86e29269028 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f92f5257bcc4e048eeadc7ae4981123', 'request_id': 'req-6def1fae-30c8-486c-bd33-d86e29269028'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:59.294099 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6def1fae-30c8-486c-bd33-d86e29269028 admin admin] The token cache is being invalidate because project 9f92f5257bcc4e048eeadc7ae4981123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:43:59.294303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6def1fae-30c8-486c-bd33-d86e29269028 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:59.295100 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-6def1fae-30c8-486c-bd33-d86e29269028 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:59.296782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1360/2731] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:43:59 2022] DELETE /identity/v3/projects/9f92f5257bcc4e048eeadc7ae4981123 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:59.301050 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def1fae-30c8-486c-bd33-d86e29269028 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:59.302876 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6def1fae-30c8-486c-bd33-d86e29269028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:59.304555 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-7167acf4-8fd5-4fd6-b640-98674366a053 None None] MFA Rules not processed for user `07d6bffc69674103b4c6540598a780b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:43:59.305770 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7167acf4-8fd5-4fd6-b640-98674366a053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:59.337997 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:59.374035 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1372/2732] 10.222.0.74 () {58 vars in 1009 bytes} [Wed Jul 27 16:43:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:43:59.387967 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167acf4-8fd5-4fd6-b640-98674366a053 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:59.389745 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7167acf4-8fd5-4fd6-b640-98674366a053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:43:59.408575 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def1fae-30c8-486c-bd33-d86e29269028 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:59.414102 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46920ca9-f8f8-47b7-b419-51709379e56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:59.414803 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46920ca9-f8f8-47b7-b419-51709379e56d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:59.415039 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46920ca9-f8f8-47b7-b419-51709379e56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:59.415315 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46920ca9-f8f8-47b7-b419-51709379e56d admin admin] PATH_INFO: `/v3/users/bd93e6ae088b4afaa317bd68a281463e` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:59.416513 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46920ca9-f8f8-47b7-b419-51709379e56d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd93e6ae088b4afaa317bd68a281463e)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:59.418547 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46920ca9-f8f8-47b7-b419-51709379e56d admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:59.473297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167acf4-8fd5-4fd6-b640-98674366a053 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:59.473761 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-46920ca9-f8f8-47b7-b419-51709379e56d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd93e6ae088b4afaa317bd68a281463e'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:59.481406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7620a4-78c2-43e7-a868-494293df6cea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb45ded9f75e4fa2973b564f83bdfaf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:59.481629 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-46920ca9-f8f8-47b7-b419-51709379e56d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd93e6ae088b4afaa317bd68a281463e'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:59.482125 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7620a4-78c2-43e7-a868-494293df6cea service swift] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:59.482550 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7620a4-78c2-43e7-a868-494293df6cea service swift] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:59.482828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7620a4-78c2-43e7-a868-494293df6cea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:59.490388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-46920ca9-f8f8-47b7-b419-51709379e56d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd93e6ae088b4afaa317bd68a281463e'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:59.494202 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7620a4-78c2-43e7-a868-494293df6cea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:59.497476 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7620a4-78c2-43e7-a868-494293df6cea service swift] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:43:59.507198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1373/2733] 10.222.0.74 () {60 vars in 1384 bytes} [Wed Jul 27 16:43:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 16:43:59.515385 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-46920ca9-f8f8-47b7-b419-51709379e56d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd93e6ae088b4afaa317bd68a281463e'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:59.516176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-46920ca9-f8f8-47b7-b419-51709379e56d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd93e6ae088b4afaa317bd68a281463e'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:43:59.522703 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1361/2734] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:43:59 2022] DELETE /identity/v3/users/bd93e6ae088b4afaa317bd68a281463e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:43:59.869877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7620a4-78c2-43e7-a868-494293df6cea service swift] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:43:59.884563 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7620a4-78c2-43e7-a868-494293df6cea service swift] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:43:59.900539 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:43:59.959545 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432acfef-6ad9-45d2-b817-94bdfe69eeaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:43:59.960791 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432acfef-6ad9-45d2-b817-94bdfe69eeaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:43:59.960791 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432acfef-6ad9-45d2-b817-94bdfe69eeaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:43:59.960791 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432acfef-6ad9-45d2-b817-94bdfe69eeaa admin admin] PATH_INFO: `/v3/projects/2bedc51874a14011bc47768b10c42752` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:43:59.961969 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432acfef-6ad9-45d2-b817-94bdfe69eeaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bedc51874a14011bc47768b10c42752)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:43:59.964092 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432acfef-6ad9-45d2-b817-94bdfe69eeaa admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:00.021947 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-432acfef-6ad9-45d2-b817-94bdfe69eeaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bedc51874a14011bc47768b10c42752', 'request_id': 'req-432acfef-6ad9-45d2-b817-94bdfe69eeaa'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:00.032976 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-432acfef-6ad9-45d2-b817-94bdfe69eeaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bedc51874a14011bc47768b10c42752', 'request_id': 'req-432acfef-6ad9-45d2-b817-94bdfe69eeaa'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:00.039968 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-432acfef-6ad9-45d2-b817-94bdfe69eeaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bedc51874a14011bc47768b10c42752', 'request_id': 'req-432acfef-6ad9-45d2-b817-94bdfe69eeaa'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:00.052001 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-432acfef-6ad9-45d2-b817-94bdfe69eeaa admin admin] The token cache is being invalidate because project 2bedc51874a14011bc47768b10c42752 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:44:00.052437 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-432acfef-6ad9-45d2-b817-94bdfe69eeaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:00.053344 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-432acfef-6ad9-45d2-b817-94bdfe69eeaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:00.055174 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1374/2735] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:43:59 2022] DELETE /identity/v3/projects/2bedc51874a14011bc47768b10c42752 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:44:00.061421 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf52a1b-8b4f-410b-a9b5-67f2f4440507 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:00.061860 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf52a1b-8b4f-410b-a9b5-67f2f4440507 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:00.062242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf52a1b-8b4f-410b-a9b5-67f2f4440507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:00.086441 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-fdf52a1b-8b4f-410b-a9b5-67f2f4440507 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:00.087659 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-fdf52a1b-8b4f-410b-a9b5-67f2f4440507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:00.120609 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb95a4b-57f0-414c-8c65-4f5aa258353f None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:00.120913 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb95a4b-57f0-414c-8c65-4f5aa258353f None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:00.121199 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb95a4b-57f0-414c-8c65-4f5aa258353f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:00.144156 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-7fb95a4b-57f0-414c-8c65-4f5aa258353f None None] MFA Rules not processed for user `07d6bffc69674103b4c6540598a780b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:00.145733 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7fb95a4b-57f0-414c-8c65-4f5aa258353f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:00.161521 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1362/2736] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:44:00.169515 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cef82-960f-4e62-b83b-5de245d260ab None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:00.170172 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cef82-960f-4e62-b83b-5de245d260ab None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:00.170674 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cef82-960f-4e62-b83b-5de245d260ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:00.198969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-5b0cef82-960f-4e62-b83b-5de245d260ab None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:00.201906 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-5b0cef82-960f-4e62-b83b-5de245d260ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:00.212040 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1363/2737] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:44:00.216035 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0cef82-960f-4e62-b83b-5de245d260ab None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:00.226138 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0cef82-960f-4e62-b83b-5de245d260ab None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:00.231619 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1de5b6-98c5-4b6e-9e7a-49b753110b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:00.232271 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1de5b6-98c5-4b6e-9e7a-49b753110b08 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:00.232552 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1de5b6-98c5-4b6e-9e7a-49b753110b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:00.233051 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1de5b6-98c5-4b6e-9e7a-49b753110b08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:00.233909 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1de5b6-98c5-4b6e-9e7a-49b753110b08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:44:00.234220 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1de5b6-98c5-4b6e-9e7a-49b753110b08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:00.234816 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1375/2738] 10.222.0.74 () {58 vars in 967 bytes} [Wed Jul 27 16:44:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:44:00.235828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1de5b6-98c5-4b6e-9e7a-49b753110b08 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:00.250292 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb95a4b-57f0-414c-8c65-4f5aa258353f None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:00.254924 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1364/2739] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:44:00.256484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1de5b6-98c5-4b6e-9e7a-49b753110b08 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:00.257178 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7fb95a4b-57f0-414c-8c65-4f5aa258353f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:00.272384 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1de5b6-98c5-4b6e-9e7a-49b753110b08 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:00.278182 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0222f-97d9-4244-a1ae-7fc563fa8280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:00.279601 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0222f-97d9-4244-a1ae-7fc563fa8280 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:00.280061 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0222f-97d9-4244-a1ae-7fc563fa8280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:00.280491 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0222f-97d9-4244-a1ae-7fc563fa8280 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:00.281353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea0222f-97d9-4244-a1ae-7fc563fa8280 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:00.284538 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea0222f-97d9-4244-a1ae-7fc563fa8280 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:00.327088 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1365/2740] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:44:00 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:44:00.327994 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0222f-97d9-4244-a1ae-7fc563fa8280 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:00.329594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-cea0222f-97d9-4244-a1ae-7fc563fa8280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:00.334948 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:00.347338 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:00.370312 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb95a4b-57f0-414c-8c65-4f5aa258353f None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:00.409364 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0222f-97d9-4244-a1ae-7fc563fa8280 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:00.425851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759eb6d-3012-4987-b2cf-e4bd45f2e270 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:00.425851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759eb6d-3012-4987-b2cf-e4bd45f2e270 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:00.425851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759eb6d-3012-4987-b2cf-e4bd45f2e270 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:00.425851 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759eb6d-3012-4987-b2cf-e4bd45f2e270 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:00.426240 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-3759eb6d-3012-4987-b2cf-e4bd45f2e270 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:00.459703 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3a876-e650-476f-ad45-8e9e3f9f0b3f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb45ded9f75e4fa2973b564f83bdfaf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:00.460326 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3a876-e650-476f-ad45-8e9e3f9f0b3f service swift] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:00.460606 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3a876-e650-476f-ad45-8e9e3f9f0b3f service swift] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:00.460880 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3a876-e650-476f-ad45-8e9e3f9f0b3f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:00.500923 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759eb6d-3012-4987-b2cf-e4bd45f2e270 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:00.503260 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759eb6d-3012-4987-b2cf-e4bd45f2e270 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:00.515645 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1366/2741] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:44:00 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:44:00.516651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759eb6d-3012-4987-b2cf-e4bd45f2e270 service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:00.545895 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d3a876-e650-476f-ad45-8e9e3f9f0b3f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:00.547479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d3a876-e650-476f-ad45-8e9e3f9f0b3f service swift] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:00.565273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1376/2742] 10.222.0.74 () {60 vars in 1384 bytes} [Wed Jul 27 16:44:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 16:44:00.629699 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759eb6d-3012-4987-b2cf-e4bd45f2e270 service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:00.635927 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fbd178-c05e-4d26-80cd-66b0a37a3a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:00.636585 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbd178-c05e-4d26-80cd-66b0a37a3a5a admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:00.636902 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbd178-c05e-4d26-80cd-66b0a37a3a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:00.637264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbd178-c05e-4d26-80cd-66b0a37a3a5a admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:00.637930 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fbd178-c05e-4d26-80cd-66b0a37a3a5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:00.639902 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fbd178-c05e-4d26-80cd-66b0a37a3a5a admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:00.674860 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1367/2743] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:44:00 2022] POST /identity/v3/users => generated 287 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:44:00.678832 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3a876-e650-476f-ad45-8e9e3f9f0b3f service swift] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:00.692290 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3a876-e650-476f-ad45-8e9e3f9f0b3f service swift] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:00.699374 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8111ec-3b2d-4436-80a9-df62269c1d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:00.700104 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8111ec-3b2d-4436-80a9-df62269c1d64 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:00.700394 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8111ec-3b2d-4436-80a9-df62269c1d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:00.700672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8111ec-3b2d-4436-80a9-df62269c1d64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:00.701839 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8111ec-3b2d-4436-80a9-df62269c1d64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:00.703181 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8111ec-3b2d-4436-80a9-df62269c1d64 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:00.720588 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1377/2744] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:44:00.722048 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fbd178-c05e-4d26-80cd-66b0a37a3a5a admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:00.737236 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fbd178-c05e-4d26-80cd-66b0a37a3a5a admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:00.757600 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958613d-b91b-4a71-b415-a0329bf2c92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:00.758244 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958613d-b91b-4a71-b415-a0329bf2c92e admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:00.758534 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958613d-b91b-4a71-b415-a0329bf2c92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:00.758814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958613d-b91b-4a71-b415-a0329bf2c92e admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:00.759522 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958613d-b91b-4a71-b415-a0329bf2c92e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:00.761307 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958613d-b91b-4a71-b415-a0329bf2c92e admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:00.777318 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1368/2745] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:44:00.781850 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8111ec-3b2d-4436-80a9-df62269c1d64 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:00.799135 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8111ec-3b2d-4436-80a9-df62269c1d64 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:00.818164 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f89a3-ed73-49c3-88e1-9ad14a289d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:00.818796 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f89a3-ed73-49c3-88e1-9ad14a289d07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:00.819076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f89a3-ed73-49c3-88e1-9ad14a289d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:00.819349 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f89a3-ed73-49c3-88e1-9ad14a289d07 admin admin] PATH_INFO: `/v3/projects/be3d0232760545aeb11555505024a388/users/ab406f01a085493292640020b9ba73f8/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:00.858115 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f89a3-ed73-49c3-88e1-9ad14a289d07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be3d0232760545aeb11555505024a388, user_id=ab406f01a085493292640020b9ba73f8, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:00.860840 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f89a3-ed73-49c3-88e1-9ad14a289d07 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:00.892949 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1378/2746] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:44:00 2022] PUT /identity/v3/projects/be3d0232760545aeb11555505024a388/users/ab406f01a085493292640020b9ba73f8/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:44:00.900871 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1235bdb-48ef-4f43-9c0f-e6c1406c5f70 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:00.901208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1235bdb-48ef-4f43-9c0f-e6c1406c5f70 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:00.901489 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1235bdb-48ef-4f43-9c0f-e6c1406c5f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:00.907253 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9d14b-5e3f-4f31-b888-3b5e2e032d4d None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:00.908245 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9d14b-5e3f-4f31-b888-3b5e2e032d4d None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:00.908245 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9d14b-5e3f-4f31-b888-3b5e2e032d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:00.941811 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-73a9d14b-5e3f-4f31-b888-3b5e2e032d4d None None] MFA Rules not processed for user `07d6bffc69674103b4c6540598a780b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:00.943791 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-f1235bdb-48ef-4f43-9c0f-e6c1406c5f70 None None] MFA Rules not processed for user `ab406f01a085493292640020b9ba73f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:00.944638 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-73a9d14b-5e3f-4f31-b888-3b5e2e032d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:00.945684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-f1235bdb-48ef-4f43-9c0f-e6c1406c5f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:00.976080 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:01.047441 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1369/2747] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:01.049020 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1379/2748] 10.222.0.74 () {58 vars in 1009 bytes} [Wed Jul 27 16:44:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:44:01.058868 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1235bdb-48ef-4f43-9c0f-e6c1406c5f70 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:01.064522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9d14b-5e3f-4f31-b888-3b5e2e032d4d None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:01.069988 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:01.123732 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1235bdb-48ef-4f43-9c0f-e6c1406c5f70 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:01.127570 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22798738-1501-4574-9648-567e86c36ff9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:01.128195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22798738-1501-4574-9648-567e86c36ff9 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:01.128477 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22798738-1501-4574-9648-567e86c36ff9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:01.128753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22798738-1501-4574-9648-567e86c36ff9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:01.138302 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22798738-1501-4574-9648-567e86c36ff9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:01.139754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22798738-1501-4574-9648-567e86c36ff9 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:01.142994 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9d14b-5e3f-4f31-b888-3b5e2e032d4d None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:01.148036 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1346e51-0ef5-4d3b-bd9f-5fb574b199dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb45ded9f75e4fa2973b564f83bdfaf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:01.148700 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1346e51-0ef5-4d3b-bd9f-5fb574b199dd service swift] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:01.148884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1370/2749] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:44:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:44:01.149010 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1346e51-0ef5-4d3b-bd9f-5fb574b199dd service swift] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:01.149313 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1346e51-0ef5-4d3b-bd9f-5fb574b199dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:01.160591 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1346e51-0ef5-4d3b-bd9f-5fb574b199dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:01.162358 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1346e51-0ef5-4d3b-bd9f-5fb574b199dd service swift] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:01.171514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22798738-1501-4574-9648-567e86c36ff9 service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:01.175251 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1380/2750] 10.222.0.74 () {60 vars in 1384 bytes} [Wed Jul 27 16:44:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 27 16:44:01.244073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22798738-1501-4574-9648-567e86c36ff9 service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:01.249337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f51f58-ead2-4d28-9390-dea658e239ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:01.250114 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f51f58-ead2-4d28-9390-dea658e239ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:01.250284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f51f58-ead2-4d28-9390-dea658e239ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:01.250543 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f51f58-ead2-4d28-9390-dea658e239ce admin admin] PATH_INFO: `/v3/users/ab406f01a085493292640020b9ba73f8` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:01.252308 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f51f58-ead2-4d28-9390-dea658e239ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab406f01a085493292640020b9ba73f8)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:01.254459 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f51f58-ead2-4d28-9390-dea658e239ce admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:01.311226 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-c2f51f58-ead2-4d28-9390-dea658e239ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab406f01a085493292640020b9ba73f8'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:01.321111 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-c2f51f58-ead2-4d28-9390-dea658e239ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab406f01a085493292640020b9ba73f8'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:01.330104 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-c2f51f58-ead2-4d28-9390-dea658e239ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab406f01a085493292640020b9ba73f8'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:01.332103 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:01.346934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-c2f51f58-ead2-4d28-9390-dea658e239ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab406f01a085493292640020b9ba73f8'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:01.348003 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-c2f51f58-ead2-4d28-9390-dea658e239ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab406f01a085493292640020b9ba73f8'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:01.358776 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1371/2751] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:44:01 2022] DELETE /identity/v3/users/ab406f01a085493292640020b9ba73f8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:44:01.363543 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719ce83-2254-4927-9ee5-bbc666a3ec35 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:01.365026 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719ce83-2254-4927-9ee5-bbc666a3ec35 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:01.365308 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719ce83-2254-4927-9ee5-bbc666a3ec35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:01.386453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-3719ce83-2254-4927-9ee5-bbc666a3ec35 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:01.388178 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-3719ce83-2254-4927-9ee5-bbc666a3ec35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:01.471749 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1381/2752] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:01.482789 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f51f58-ead2-4d28-9390-dea658e239ce admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:01.484332 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-c2f51f58-ead2-4d28-9390-dea658e239ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:01.597081 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645bf64-a2d0-4fbd-84a8-1aea1968c22b None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:01.597372 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645bf64-a2d0-4fbd-84a8-1aea1968c22b None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:01.597652 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645bf64-a2d0-4fbd-84a8-1aea1968c22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:01.628137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-2645bf64-a2d0-4fbd-84a8-1aea1968c22b None None] MFA Rules not processed for user `07d6bffc69674103b4c6540598a780b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:01.630093 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-2645bf64-a2d0-4fbd-84a8-1aea1968c22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:01.678142 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f51f58-ead2-4d28-9390-dea658e239ce admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:01.686676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041f6e0-612a-4704-b0b7-8f710d0f45fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:01.687317 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041f6e0-612a-4704-b0b7-8f710d0f45fa service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:01.687604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041f6e0-612a-4704-b0b7-8f710d0f45fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:01.687884 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041f6e0-612a-4704-b0b7-8f710d0f45fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:01.707372 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041f6e0-612a-4704-b0b7-8f710d0f45fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:01.709469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1382/2753] 10.222.0.74 () {58 vars in 967 bytes} [Wed Jul 27 16:44:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:01.710044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041f6e0-612a-4704-b0b7-8f710d0f45fa service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:01.724855 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645bf64-a2d0-4fbd-84a8-1aea1968c22b None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:01.726531 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-2645bf64-a2d0-4fbd-84a8-1aea1968c22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:01.741671 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1372/2754] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:44:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:44:01.786181 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:01.837505 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645bf64-a2d0-4fbd-84a8-1aea1968c22b None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:01.845775 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153496ed-5a04-4046-a83c-3042f8bb9631 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb45ded9f75e4fa2973b564f83bdfaf2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:01.846877 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153496ed-5a04-4046-a83c-3042f8bb9631 service swift] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:01.847377 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153496ed-5a04-4046-a83c-3042f8bb9631 service swift] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:01.847856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153496ed-5a04-4046-a83c-3042f8bb9631 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:01.862885 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153496ed-5a04-4046-a83c-3042f8bb9631 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:01.864833 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153496ed-5a04-4046-a83c-3042f8bb9631 service swift] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:01.876730 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1383/2755] 10.222.0.74 () {60 vars in 1384 bytes} [Wed Jul 27 16:44:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 27 16:44:02.107813 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041f6e0-612a-4704-b0b7-8f710d0f45fa service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:02.109453 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-a041f6e0-612a-4704-b0b7-8f710d0f45fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:02.176194 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041f6e0-612a-4704-b0b7-8f710d0f45fa service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:02.182343 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d409a24-9092-466a-a89e-a8b0b9c7c6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:02.182987 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d409a24-9092-466a-a89e-a8b0b9c7c6fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:02.183506 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d409a24-9092-466a-a89e-a8b0b9c7c6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:02.188580 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d409a24-9092-466a-a89e-a8b0b9c7c6fd admin admin] PATH_INFO: `/v3/projects/be3d0232760545aeb11555505024a388` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:02.190024 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d409a24-9092-466a-a89e-a8b0b9c7c6fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=be3d0232760545aeb11555505024a388)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:02.192926 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d409a24-9092-466a-a89e-a8b0b9c7c6fd admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:02.255666 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-1d409a24-9092-466a-a89e-a8b0b9c7c6fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be3d0232760545aeb11555505024a388', 'request_id': 'req-1d409a24-9092-466a-a89e-a8b0b9c7c6fd'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:02.265347 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-1d409a24-9092-466a-a89e-a8b0b9c7c6fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be3d0232760545aeb11555505024a388', 'request_id': 'req-1d409a24-9092-466a-a89e-a8b0b9c7c6fd'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:02.275818 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-1d409a24-9092-466a-a89e-a8b0b9c7c6fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be3d0232760545aeb11555505024a388', 'request_id': 'req-1d409a24-9092-466a-a89e-a8b0b9c7c6fd'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:02.293674 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-1d409a24-9092-466a-a89e-a8b0b9c7c6fd admin admin] The token cache is being invalidate because project be3d0232760545aeb11555505024a388 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:44:02.294228 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-1d409a24-9092-466a-a89e-a8b0b9c7c6fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:02.295280 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-1d409a24-9092-466a-a89e-a8b0b9c7c6fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:02.297383 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1373/2756] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:44:02 2022] DELETE /identity/v3/projects/be3d0232760545aeb11555505024a388 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:44:02.302559 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8365f4-0e31-4e28-b356-5231131e869c None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:02.302923 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8365f4-0e31-4e28-b356-5231131e869c None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:02.303211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8365f4-0e31-4e28-b356-5231131e869c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:02.347905 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-6a8365f4-0e31-4e28-b356-5231131e869c None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:02.351840 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-6a8365f4-0e31-4e28-b356-5231131e869c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:02.444109 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1384/2757] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:44:02.445231 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a09f0-439b-466c-bb27-b7537648fba5 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:02.445532 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a09f0-439b-466c-bb27-b7537648fba5 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:02.445917 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a09f0-439b-466c-bb27-b7537648fba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:02.495364 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-6f4a09f0-439b-466c-bb27-b7537648fba5 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:02.497656 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-6f4a09f0-439b-466c-bb27-b7537648fba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:02.513726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1374/2758] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:02.518597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8365f4-0e31-4e28-b356-5231131e869c None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:02.530279 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8365f4-0e31-4e28-b356-5231131e869c None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:02.551384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f6c40-e317-452c-99cc-92efede32ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:02.551384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f6c40-e317-452c-99cc-92efede32ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:02.551384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f6c40-e317-452c-99cc-92efede32ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:02.551384 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f6c40-e317-452c-99cc-92efede32ee6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:02.551728 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f6c40-e317-452c-99cc-92efede32ee6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:44:02.551728 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f6c40-e317-452c-99cc-92efede32ee6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:02.551728 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f6c40-e317-452c-99cc-92efede32ee6 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:02.558802 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1385/2759] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:44:02.564620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a09f0-439b-466c-bb27-b7537648fba5 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:02.576883 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a09f0-439b-466c-bb27-b7537648fba5 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:02.588281 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a34c1b-91c1-4a06-87a4-80137e9a9676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:02.588946 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a34c1b-91c1-4a06-87a4-80137e9a9676 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:02.589236 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a34c1b-91c1-4a06-87a4-80137e9a9676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:02.589506 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a34c1b-91c1-4a06-87a4-80137e9a9676 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:02.590226 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a34c1b-91c1-4a06-87a4-80137e9a9676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:02.592216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a34c1b-91c1-4a06-87a4-80137e9a9676 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:02.645207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1375/2760] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:44:02 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:44:02.646029 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f6c40-e317-452c-99cc-92efede32ee6 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:02.657898 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f6c40-e317-452c-99cc-92efede32ee6 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:02.749934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b333a94-d347-4ed0-975b-8f4fd247df10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:02.750313 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b333a94-d347-4ed0-975b-8f4fd247df10 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:02.750676 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b333a94-d347-4ed0-975b-8f4fd247df10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:02.751028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b333a94-d347-4ed0-975b-8f4fd247df10 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:02.751945 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b333a94-d347-4ed0-975b-8f4fd247df10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:02.755394 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b333a94-d347-4ed0-975b-8f4fd247df10 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:02.783771 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1386/2761] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:44:02 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:44:02.785899 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a34c1b-91c1-4a06-87a4-80137e9a9676 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:02.817905 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a34c1b-91c1-4a06-87a4-80137e9a9676 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:02.832686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09449b9-efbd-4c0b-9ccd-8a4f8a7acb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:02.833370 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09449b9-efbd-4c0b-9ccd-8a4f8a7acb9f admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:02.833686 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09449b9-efbd-4c0b-9ccd-8a4f8a7acb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:02.833965 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09449b9-efbd-4c0b-9ccd-8a4f8a7acb9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:02.834706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09449b9-efbd-4c0b-9ccd-8a4f8a7acb9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:02.836399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09449b9-efbd-4c0b-9ccd-8a4f8a7acb9f admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:02.863651 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1376/2762] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:44:02.868421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b333a94-d347-4ed0-975b-8f4fd247df10 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:02.883020 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b333a94-d347-4ed0-975b-8f4fd247df10 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:02.897117 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02bccd0-dbaf-4945-93e4-c826c49c2a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:02.897762 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bccd0-dbaf-4945-93e4-c826c49c2a50 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:02.898044 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bccd0-dbaf-4945-93e4-c826c49c2a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:02.898323 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bccd0-dbaf-4945-93e4-c826c49c2a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:02.900108 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02bccd0-dbaf-4945-93e4-c826c49c2a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:02.900825 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02bccd0-dbaf-4945-93e4-c826c49c2a50 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:02.959979 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1387/2763] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:44:02.963694 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09449b9-efbd-4c0b-9ccd-8a4f8a7acb9f admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:02.979844 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09449b9-efbd-4c0b-9ccd-8a4f8a7acb9f admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:02.994124 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c7e59-b5e9-43fb-b90f-49559993ac42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:02.994825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c7e59-b5e9-43fb-b90f-49559993ac42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:02.995116 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c7e59-b5e9-43fb-b90f-49559993ac42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:02.995401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c7e59-b5e9-43fb-b90f-49559993ac42 admin admin] PATH_INFO: `/v3/projects/f4291557abaf483d8cbfbbc26e14eb31/users/7a5de1a4678a42ca8d716ad084e972a2/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:03.002291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:03.043446 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c7e59-b5e9-43fb-b90f-49559993ac42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4291557abaf483d8cbfbbc26e14eb31, user_id=7a5de1a4678a42ca8d716ad084e972a2, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:03.047517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c7e59-b5e9-43fb-b90f-49559993ac42 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:03.105762 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6181a-90ad-4618-ab25-8f40ed917552 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:03.105762 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6181a-90ad-4618-ab25-8f40ed917552 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:03.105762 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6181a-90ad-4618-ab25-8f40ed917552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:03.109258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1377/2764] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:44:02 2022] PUT /identity/v3/projects/f4291557abaf483d8cbfbbc26e14eb31/users/7a5de1a4678a42ca8d716ad084e972a2/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:44:03.172503 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-edd6181a-90ad-4618-ab25-8f40ed917552 None None] MFA Rules not processed for user `7a5de1a4678a42ca8d716ad084e972a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:03.173800 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-edd6181a-90ad-4618-ab25-8f40ed917552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:03.204077 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:03.252215 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1388/2765] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:44:03.257434 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c7e59-b5e9-43fb-b90f-49559993ac42 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:03.393477 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c7e59-b5e9-43fb-b90f-49559993ac42 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:03.409824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9f921-f9a0-492c-9884-ce6cb2139086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:03.414625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9f921-f9a0-492c-9884-ce6cb2139086 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:03.415221 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9f921-f9a0-492c-9884-ce6cb2139086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:03.415514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9f921-f9a0-492c-9884-ce6cb2139086 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:03.416212 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9f921-f9a0-492c-9884-ce6cb2139086 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:03.420239 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9f921-f9a0-492c-9884-ce6cb2139086 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:03.494533 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1378/2766] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:44:03 2022] POST /identity/v3/projects => generated 383 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:44:03.497761 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6181a-90ad-4618-ab25-8f40ed917552 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:03.517557 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6181a-90ad-4618-ab25-8f40ed917552 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:03.602638 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748968c-8af7-4936-bc49-3105b1f37cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:03.603311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748968c-8af7-4936-bc49-3105b1f37cab admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:03.604146 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748968c-8af7-4936-bc49-3105b1f37cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:03.604425 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748968c-8af7-4936-bc49-3105b1f37cab admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:03.605112 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748968c-8af7-4936-bc49-3105b1f37cab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:03.608180 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6748968c-8af7-4936-bc49-3105b1f37cab admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:03.611114 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9f921-f9a0-492c-9884-ce6cb2139086 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:03.614256 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-e3a9f921-f9a0-492c-9884-ce6cb2139086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:03.633873 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9f921-f9a0-492c-9884-ce6cb2139086 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:03.639836 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1389/2767] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:44:03 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:44:03.644083 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf474b-8270-4621-ac2d-c05d80f3d6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:03.644537 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748968c-8af7-4936-bc49-3105b1f37cab admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:03.644941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf474b-8270-4621-ac2d-c05d80f3d6b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:03.645249 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf474b-8270-4621-ac2d-c05d80f3d6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:03.645671 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf474b-8270-4621-ac2d-c05d80f3d6b2 admin admin] PATH_INFO: `/v3/users/6e5020300565412f9a251e471aeb17b9` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:03.647739 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf474b-8270-4621-ac2d-c05d80f3d6b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e5020300565412f9a251e471aeb17b9)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:03.649737 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf474b-8270-4621-ac2d-c05d80f3d6b2 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:03.667693 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748968c-8af7-4936-bc49-3105b1f37cab admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:03.674225 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b555a-ca82-45b8-a4f5-7300c0986169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:03.674845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b555a-ca82-45b8-a4f5-7300c0986169 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:03.675122 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b555a-ca82-45b8-a4f5-7300c0986169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:03.675393 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b555a-ca82-45b8-a4f5-7300c0986169 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:03.676172 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b555a-ca82-45b8-a4f5-7300c0986169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:03.679194 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b555a-ca82-45b8-a4f5-7300c0986169 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:03.700205 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1390/2768] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:44:03.703637 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b555a-ca82-45b8-a4f5-7300c0986169 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:03.719734 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b555a-ca82-45b8-a4f5-7300c0986169 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:03.726025 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-dcaf474b-8270-4621-ac2d-c05d80f3d6b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e5020300565412f9a251e471aeb17b9'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:03.727110 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89a0d0-9437-4f6a-8a44-3b46fa502a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:03.727864 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89a0d0-9437-4f6a-8a44-3b46fa502a39 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:03.728154 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89a0d0-9437-4f6a-8a44-3b46fa502a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:03.728422 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89a0d0-9437-4f6a-8a44-3b46fa502a39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:03.729480 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89a0d0-9437-4f6a-8a44-3b46fa502a39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:03.731995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89a0d0-9437-4f6a-8a44-3b46fa502a39 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:03.736327 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-dcaf474b-8270-4621-ac2d-c05d80f3d6b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e5020300565412f9a251e471aeb17b9'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:03.747207 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1391/2769] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:44:03.750162 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-dcaf474b-8270-4621-ac2d-c05d80f3d6b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e5020300565412f9a251e471aeb17b9'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:03.751128 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89a0d0-9437-4f6a-8a44-3b46fa502a39 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:03.769654 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-dcaf474b-8270-4621-ac2d-c05d80f3d6b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e5020300565412f9a251e471aeb17b9'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:03.770504 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89a0d0-9437-4f6a-8a44-3b46fa502a39 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:03.770707 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-dcaf474b-8270-4621-ac2d-c05d80f3d6b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e5020300565412f9a251e471aeb17b9'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:03.776983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5f8bb-8c56-4850-b0ee-85ba4bf5969a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:03.778746 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5f8bb-8c56-4850-b0ee-85ba4bf5969a admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:03.779273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5f8bb-8c56-4850-b0ee-85ba4bf5969a admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:03.779782 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1379/2770] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:44:03 2022] DELETE /identity/v3/users/6e5020300565412f9a251e471aeb17b9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:44:03.780047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5f8bb-8c56-4850-b0ee-85ba4bf5969a admin admin] PATH_INFO: `/v3/projects/6d8d59d2b433466b897ce8ecd37df11b/users/dae35e04bf3941b38829b35ef9a31a2f/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:03.801330 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a5f8bb-8c56-4850-b0ee-85ba4bf5969a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d8d59d2b433466b897ce8ecd37df11b, user_id=dae35e04bf3941b38829b35ef9a31a2f, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:03.805129 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a5f8bb-8c56-4850-b0ee-85ba4bf5969a admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:03.823015 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1392/2771] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:44:03 2022] PUT /identity/v3/projects/6d8d59d2b433466b897ce8ecd37df11b/users/dae35e04bf3941b38829b35ef9a31a2f/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:44:03.894661 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50b6db-9f30-4bdd-b021-d1f6ca617a02 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:03.895003 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50b6db-9f30-4bdd-b021-d1f6ca617a02 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:03.895264 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50b6db-9f30-4bdd-b021-d1f6ca617a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:03.934728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-ba50b6db-9f30-4bdd-b021-d1f6ca617a02 None None] MFA Rules not processed for user `dae35e04bf3941b38829b35ef9a31a2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:03.936187 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ba50b6db-9f30-4bdd-b021-d1f6ca617a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:03.974256 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:03.986237 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5f8bb-8c56-4850-b0ee-85ba4bf5969a admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:03.989599 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e1a5f8bb-8c56-4850-b0ee-85ba4bf5969a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:04.036934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1380/2772] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:04.041611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50b6db-9f30-4bdd-b021-d1f6ca617a02 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:04.043293 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ba50b6db-9f30-4bdd-b021-d1f6ca617a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:04.081253 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5f8bb-8c56-4850-b0ee-85ba4bf5969a admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:04.086473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3a2f1-f5a6-4178-9f70-f5892dbcdd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:04.087470 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3a2f1-f5a6-4178-9f70-f5892dbcdd77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:04.087977 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3a2f1-f5a6-4178-9f70-f5892dbcdd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:04.088460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3a2f1-f5a6-4178-9f70-f5892dbcdd77 admin admin] PATH_INFO: `/v3/projects/95d1afc492b44c07800f8d7288dac741` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:04.090735 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3a2f1-f5a6-4178-9f70-f5892dbcdd77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95d1afc492b44c07800f8d7288dac741)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:04.092853 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3a2f1-f5a6-4178-9f70-f5892dbcdd77 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:04.109330 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:04.124443 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50b6db-9f30-4bdd-b021-d1f6ca617a02 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:04.129302 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdffd65-829f-499e-a690-756708c0a0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:04.130343 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdffd65-829f-499e-a690-756708c0a0b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:04.130839 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdffd65-829f-499e-a690-756708c0a0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:04.131305 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdffd65-829f-499e-a690-756708c0a0b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:04.132162 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdffd65-829f-499e-a690-756708c0a0b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:04.134509 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdffd65-829f-499e-a690-756708c0a0b6 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:04.175598 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-3dc3a2f1-f5a6-4178-9f70-f5892dbcdd77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95d1afc492b44c07800f8d7288dac741', 'request_id': 'req-3dc3a2f1-f5a6-4178-9f70-f5892dbcdd77'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:04.182814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1381/2773] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:44:04 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:44:04.184786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-3dc3a2f1-f5a6-4178-9f70-f5892dbcdd77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95d1afc492b44c07800f8d7288dac741', 'request_id': 'req-3dc3a2f1-f5a6-4178-9f70-f5892dbcdd77'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:04.186911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdffd65-829f-499e-a690-756708c0a0b6 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:04.192469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-3dc3a2f1-f5a6-4178-9f70-f5892dbcdd77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95d1afc492b44c07800f8d7288dac741', 'request_id': 'req-3dc3a2f1-f5a6-4178-9f70-f5892dbcdd77'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:04.206495 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-3dc3a2f1-f5a6-4178-9f70-f5892dbcdd77 admin admin] The token cache is being invalidate because project 95d1afc492b44c07800f8d7288dac741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:44:04.206824 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-3dc3a2f1-f5a6-4178-9f70-f5892dbcdd77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:04.207641 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-3dc3a2f1-f5a6-4178-9f70-f5892dbcdd77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:04.209843 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1393/2774] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:44:03 2022] DELETE /identity/v3/projects/95d1afc492b44c07800f8d7288dac741 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:44:04.214127 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3a2f1-f5a6-4178-9f70-f5892dbcdd77 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:04.216707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-3dc3a2f1-f5a6-4178-9f70-f5892dbcdd77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:04.256129 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdffd65-829f-499e-a690-756708c0a0b6 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:04.262291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e662f04-c3f4-4543-a0d8-e1ef8dda9e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:04.262941 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e662f04-c3f4-4543-a0d8-e1ef8dda9e19 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:04.263322 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e662f04-c3f4-4543-a0d8-e1ef8dda9e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:04.264267 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e662f04-c3f4-4543-a0d8-e1ef8dda9e19 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:04.265044 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e662f04-c3f4-4543-a0d8-e1ef8dda9e19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:04.267257 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e662f04-c3f4-4543-a0d8-e1ef8dda9e19 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:04.289058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3a2f1-f5a6-4178-9f70-f5892dbcdd77 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:04.294713 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c22410-8d92-42b7-a09c-7dfbccbe8a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:04.295466 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c22410-8d92-42b7-a09c-7dfbccbe8a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:04.295750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c22410-8d92-42b7-a09c-7dfbccbe8a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:04.296119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c22410-8d92-42b7-a09c-7dfbccbe8a82 admin admin] PATH_INFO: `/v3/users/684e37a8185a4199b95d407f9d979a06` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:04.297750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c22410-8d92-42b7-a09c-7dfbccbe8a82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=684e37a8185a4199b95d407f9d979a06)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:04.299974 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c22410-8d92-42b7-a09c-7dfbccbe8a82 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:04.309033 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1382/2775] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:44:04 2022] POST /identity/v3/users => generated 298 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:44:04.313052 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e662f04-c3f4-4543-a0d8-e1ef8dda9e19 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:04.315125 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-2e662f04-c3f4-4543-a0d8-e1ef8dda9e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:04.335103 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e662f04-c3f4-4543-a0d8-e1ef8dda9e19 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:04.342577 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417daf2c-e4ab-4155-a0ec-886b08bf38da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:04.343212 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417daf2c-e4ab-4155-a0ec-886b08bf38da admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:04.343500 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417daf2c-e4ab-4155-a0ec-886b08bf38da admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:04.343784 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417daf2c-e4ab-4155-a0ec-886b08bf38da admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:04.344484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417daf2c-e4ab-4155-a0ec-886b08bf38da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:04.346418 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417daf2c-e4ab-4155-a0ec-886b08bf38da admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:04.354969 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-74c22410-8d92-42b7-a09c-7dfbccbe8a82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '684e37a8185a4199b95d407f9d979a06'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:04.357205 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1383/2776] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:44:04.361604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417daf2c-e4ab-4155-a0ec-886b08bf38da admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:04.366700 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-74c22410-8d92-42b7-a09c-7dfbccbe8a82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '684e37a8185a4199b95d407f9d979a06'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:04.372907 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417daf2c-e4ab-4155-a0ec-886b08bf38da admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:04.377631 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75384318-485e-4155-8fe4-fb31eb3e3c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:04.378283 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75384318-485e-4155-8fe4-fb31eb3e3c15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:04.378566 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75384318-485e-4155-8fe4-fb31eb3e3c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:04.378841 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75384318-485e-4155-8fe4-fb31eb3e3c15 admin admin] PATH_INFO: `/v3/projects/7b085ad470b2405e93c9606ba2655c23/users/f85d84dbe6b64ac8b809c4f0cb19bf20/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:04.379928 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-74c22410-8d92-42b7-a09c-7dfbccbe8a82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '684e37a8185a4199b95d407f9d979a06'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:04.386861 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-74c22410-8d92-42b7-a09c-7dfbccbe8a82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '684e37a8185a4199b95d407f9d979a06'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:04.388410 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-74c22410-8d92-42b7-a09c-7dfbccbe8a82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '684e37a8185a4199b95d407f9d979a06'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:04.394531 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1394/2777] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:44:04 2022] DELETE /identity/v3/users/684e37a8185a4199b95d407f9d979a06 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:44:04.394665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75384318-485e-4155-8fe4-fb31eb3e3c15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b085ad470b2405e93c9606ba2655c23, user_id=f85d84dbe6b64ac8b809c4f0cb19bf20, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:04.397570 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75384318-485e-4155-8fe4-fb31eb3e3c15 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:04.410466 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1384/2778] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:44:04 2022] PUT /identity/v3/projects/7b085ad470b2405e93c9606ba2655c23/users/f85d84dbe6b64ac8b809c4f0cb19bf20/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:44:04.414547 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c22410-8d92-42b7-a09c-7dfbccbe8a82 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:04.416125 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-74c22410-8d92-42b7-a09c-7dfbccbe8a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:04.438962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:04.462970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75384318-485e-4155-8fe4-fb31eb3e3c15 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:04.464733 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-75384318-485e-4155-8fe4-fb31eb3e3c15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:04.479359 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:04.495624 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c22410-8d92-42b7-a09c-7dfbccbe8a82 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:04.548010 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75384318-485e-4155-8fe4-fb31eb3e3c15 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:04.553460 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb392812-4fc1-4a5c-be0c-a9c131d95654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:04.554119 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb392812-4fc1-4a5c-be0c-a9c131d95654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:04.554403 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb392812-4fc1-4a5c-be0c-a9c131d95654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:04.554676 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb392812-4fc1-4a5c-be0c-a9c131d95654 admin admin] PATH_INFO: `/v3/users/189d57780b254f90ae9120f4c31d987a` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:04.555986 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb392812-4fc1-4a5c-be0c-a9c131d95654 admin admin] RBAC: Authorizing `identity:delete_user(user_id=189d57780b254f90ae9120f4c31d987a)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:04.557972 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb392812-4fc1-4a5c-be0c-a9c131d95654 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:04.559607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9da54-537f-4bf2-b08d-455a8d208f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:04.560627 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9da54-537f-4bf2-b08d-455a8d208f6c admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:04.560941 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9da54-537f-4bf2-b08d-455a8d208f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:04.561224 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9da54-537f-4bf2-b08d-455a8d208f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:04.562287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9da54-537f-4bf2-b08d-455a8d208f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:04.564058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9da54-537f-4bf2-b08d-455a8d208f6c admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:04.586349 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1395/2779] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:44:04.586349 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9da54-537f-4bf2-b08d-455a8d208f6c admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:04.586349 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-0af9da54-537f-4bf2-b08d-455a8d208f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:04.602220 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9da54-537f-4bf2-b08d-455a8d208f6c admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:04.607869 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307668fa-94c1-4e3c-bc28-39976119648a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:04.608528 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307668fa-94c1-4e3c-bc28-39976119648a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:04.608812 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307668fa-94c1-4e3c-bc28-39976119648a admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:04.609247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307668fa-94c1-4e3c-bc28-39976119648a admin admin] PATH_INFO: `/v3/projects/49fb5ac1be6848ec9263b3431bd87f4a` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:04.611277 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307668fa-94c1-4e3c-bc28-39976119648a admin admin] RBAC: Authorizing `identity:delete_project(project_id=49fb5ac1be6848ec9263b3431bd87f4a)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:04.613466 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307668fa-94c1-4e3c-bc28-39976119648a admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:04.626564 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-eb392812-4fc1-4a5c-be0c-a9c131d95654 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '189d57780b254f90ae9120f4c31d987a'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:04.634375 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-eb392812-4fc1-4a5c-be0c-a9c131d95654 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '189d57780b254f90ae9120f4c31d987a'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:04.642051 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-eb392812-4fc1-4a5c-be0c-a9c131d95654 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '189d57780b254f90ae9120f4c31d987a'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:04.654757 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-eb392812-4fc1-4a5c-be0c-a9c131d95654 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '189d57780b254f90ae9120f4c31d987a'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:04.655473 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-eb392812-4fc1-4a5c-be0c-a9c131d95654 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '189d57780b254f90ae9120f4c31d987a'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:04.662454 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1385/2780] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:44:04 2022] DELETE /identity/v3/users/189d57780b254f90ae9120f4c31d987a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:44:04.662581 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-307668fa-94c1-4e3c-bc28-39976119648a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49fb5ac1be6848ec9263b3431bd87f4a', 'request_id': 'req-307668fa-94c1-4e3c-bc28-39976119648a'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:04.666059 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb392812-4fc1-4a5c-be0c-a9c131d95654 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:04.668054 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-eb392812-4fc1-4a5c-be0c-a9c131d95654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:04.671092 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-307668fa-94c1-4e3c-bc28-39976119648a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49fb5ac1be6848ec9263b3431bd87f4a', 'request_id': 'req-307668fa-94c1-4e3c-bc28-39976119648a'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:04.678076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-307668fa-94c1-4e3c-bc28-39976119648a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49fb5ac1be6848ec9263b3431bd87f4a', 'request_id': 'req-307668fa-94c1-4e3c-bc28-39976119648a'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:04.697866 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-307668fa-94c1-4e3c-bc28-39976119648a admin admin] The token cache is being invalidate because project 49fb5ac1be6848ec9263b3431bd87f4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:44:04.698311 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-307668fa-94c1-4e3c-bc28-39976119648a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:04.699989 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-307668fa-94c1-4e3c-bc28-39976119648a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:04.701873 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1396/2781] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:44:04 2022] DELETE /identity/v3/projects/49fb5ac1be6848ec9263b3431bd87f4a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:44:04.706206 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f887a7-fb79-4ed1-87d6-bb80db083a2b None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:04.706547 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f887a7-fb79-4ed1-87d6-bb80db083a2b None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:04.706829 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f887a7-fb79-4ed1-87d6-bb80db083a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:04.729328 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-95f887a7-fb79-4ed1-87d6-bb80db083a2b None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:04.730777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-95f887a7-fb79-4ed1-87d6-bb80db083a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:04.776262 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb392812-4fc1-4a5c-be0c-a9c131d95654 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:04.782114 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8990d74-2928-4063-827a-a452a9383e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:04.782827 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8990d74-2928-4063-827a-a452a9383e3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:04.783116 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8990d74-2928-4063-827a-a452a9383e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:04.783397 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8990d74-2928-4063-827a-a452a9383e3d admin admin] PATH_INFO: `/v3/projects/7b085ad470b2405e93c9606ba2655c23/users/f85d84dbe6b64ac8b809c4f0cb19bf20/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:04.785868 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8990d74-2928-4063-827a-a452a9383e3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b085ad470b2405e93c9606ba2655c23, user_id=f85d84dbe6b64ac8b809c4f0cb19bf20, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:04.787587 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:04.788779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8990d74-2928-4063-827a-a452a9383e3d admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:04.802562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1386/2782] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:44:04 2022] PUT /identity/v3/projects/7b085ad470b2405e93c9606ba2655c23/users/f85d84dbe6b64ac8b809c4f0cb19bf20/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:44:04.814443 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e727eaf-1837-489a-9eb5-1e41c94dc9c4 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:04.814808 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e727eaf-1837-489a-9eb5-1e41c94dc9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:04.815019 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e727eaf-1837-489a-9eb5-1e41c94dc9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:04.822343 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1397/2783] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:04.832529 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f887a7-fb79-4ed1-87d6-bb80db083a2b None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:04.838324 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-95f887a7-fb79-4ed1-87d6-bb80db083a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:04.862665 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-0e727eaf-1837-489a-9eb5-1e41c94dc9c4 None None] MFA Rules not processed for user `f85d84dbe6b64ac8b809c4f0cb19bf20`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:04.864105 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0e727eaf-1837-489a-9eb5-1e41c94dc9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:04.892148 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:04.929512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f887a7-fb79-4ed1-87d6-bb80db083a2b None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:04.976970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1387/2784] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:04.988476 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e727eaf-1837-489a-9eb5-1e41c94dc9c4 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:04.990176 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0e727eaf-1837-489a-9eb5-1e41c94dc9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:05.003738 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d82f2-68e1-4312-8beb-2622662ee996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:05.004651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d82f2-68e1-4312-8beb-2622662ee996 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:05.004956 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d82f2-68e1-4312-8beb-2622662ee996 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:05.005236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d82f2-68e1-4312-8beb-2622662ee996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:05.058006 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e727eaf-1837-489a-9eb5-1e41c94dc9c4 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:05.062395 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93051be0-9959-4d4f-9cb5-51925e1c3d63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:05.063034 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93051be0-9959-4d4f-9cb5-51925e1c3d63 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:05.063320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93051be0-9959-4d4f-9cb5-51925e1c3d63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:05.063601 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93051be0-9959-4d4f-9cb5-51925e1c3d63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:05.065484 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-93051be0-9959-4d4f-9cb5-51925e1c3d63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:05.079732 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d82f2-68e1-4312-8beb-2622662ee996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:05.081442 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d82f2-68e1-4312-8beb-2622662ee996 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:05.093413 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1398/2785] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:44:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:44:05.136583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93051be0-9959-4d4f-9cb5-51925e1c3d63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:05.139370 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93051be0-9959-4d4f-9cb5-51925e1c3d63 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:05.159580 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1388/2786] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:44:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:44:05.418591 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d82f2-68e1-4312-8beb-2622662ee996 service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:05.420071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-8e8d82f2-68e1-4312-8beb-2622662ee996 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:05.437821 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d82f2-68e1-4312-8beb-2622662ee996 service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:05.443351 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ddb03-4b46-4cf0-84c7-a7bd87109428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:05.443966 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ddb03-4b46-4cf0-84c7-a7bd87109428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:05.444246 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ddb03-4b46-4cf0-84c7-a7bd87109428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:05.444671 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ddb03-4b46-4cf0-84c7-a7bd87109428 admin admin] PATH_INFO: `/v3/projects/6ad80139b6a04b62be6629aa767024f3` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:05.449910 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ddb03-4b46-4cf0-84c7-a7bd87109428 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ad80139b6a04b62be6629aa767024f3)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:05.452125 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ddb03-4b46-4cf0-84c7-a7bd87109428 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:05.506499 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:05.510054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-d04ddb03-4b46-4cf0-84c7-a7bd87109428 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ad80139b6a04b62be6629aa767024f3', 'request_id': 'req-d04ddb03-4b46-4cf0-84c7-a7bd87109428'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:05.519815 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-d04ddb03-4b46-4cf0-84c7-a7bd87109428 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ad80139b6a04b62be6629aa767024f3', 'request_id': 'req-d04ddb03-4b46-4cf0-84c7-a7bd87109428'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:05.525806 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-d04ddb03-4b46-4cf0-84c7-a7bd87109428 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ad80139b6a04b62be6629aa767024f3', 'request_id': 'req-d04ddb03-4b46-4cf0-84c7-a7bd87109428'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:05.537787 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-d04ddb03-4b46-4cf0-84c7-a7bd87109428 admin admin] The token cache is being invalidate because project 6ad80139b6a04b62be6629aa767024f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:44:05.538076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-d04ddb03-4b46-4cf0-84c7-a7bd87109428 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:05.538724 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-d04ddb03-4b46-4cf0-84c7-a7bd87109428 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:05.540233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1399/2787] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:44:05 2022] DELETE /identity/v3/projects/6ad80139b6a04b62be6629aa767024f3 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:44:05.545910 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261f453-72a4-40c7-8f4f-2b38bdf1047a None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:05.546451 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261f453-72a4-40c7-8f4f-2b38bdf1047a None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:05.546691 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261f453-72a4-40c7-8f4f-2b38bdf1047a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:05.568414 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-9261f453-72a4-40c7-8f4f-2b38bdf1047a None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:05.569675 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9261f453-72a4-40c7-8f4f-2b38bdf1047a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:05.630232 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1389/2788] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:05.638007 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0f817-800b-4720-a608-c37d3f8cff70 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:05.638491 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0f817-800b-4720-a608-c37d3f8cff70 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:05.638706 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0f817-800b-4720-a608-c37d3f8cff70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:05.658587 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-d1c0f817-800b-4720-a608-c37d3f8cff70 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:05.659784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d1c0f817-800b-4720-a608-c37d3f8cff70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:05.667819 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1400/2789] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:05.672599 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261f453-72a4-40c7-8f4f-2b38bdf1047a None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:05.678558 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=100 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:05.682843 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261f453-72a4-40c7-8f4f-2b38bdf1047a None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:05.689580 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694c399-4865-4532-91ba-07c49b61193a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:05.690205 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694c399-4865-4532-91ba-07c49b61193a admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:05.690495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694c399-4865-4532-91ba-07c49b61193a admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:05.690793 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694c399-4865-4532-91ba-07c49b61193a admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:05.691590 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694c399-4865-4532-91ba-07c49b61193a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:44:05.691930 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694c399-4865-4532-91ba-07c49b61193a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:05.693612 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694c399-4865-4532-91ba-07c49b61193a admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:05.705760 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1390/2790] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:44:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:44:05.710263 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0f817-800b-4720-a608-c37d3f8cff70 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:05.719539 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0f817-800b-4720-a608-c37d3f8cff70 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:05.727170 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d5a44-e393-455d-9950-78c268525a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:05.727768 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d5a44-e393-455d-9950-78c268525a10 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:05.728078 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d5a44-e393-455d-9950-78c268525a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:05.728349 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d5a44-e393-455d-9950-78c268525a10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:05.729054 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d5a44-e393-455d-9950-78c268525a10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:05.731473 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d5a44-e393-455d-9950-78c268525a10 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:05.761537 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1401/2791] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:44:05 2022] POST /identity/v3/projects => generated 395 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:44:05.766203 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694c399-4865-4532-91ba-07c49b61193a admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:05.776633 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694c399-4865-4532-91ba-07c49b61193a admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:05.836311 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cbf54-1468-4c7b-aa6a-831ef4d0be58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:05.837250 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cbf54-1468-4c7b-aa6a-831ef4d0be58 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:05.837728 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cbf54-1468-4c7b-aa6a-831ef4d0be58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:05.838003 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cbf54-1468-4c7b-aa6a-831ef4d0be58 admin admin] PATH_INFO: `/v3/users` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:05.838684 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3cbf54-1468-4c7b-aa6a-831ef4d0be58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:05.840605 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3cbf54-1468-4c7b-aa6a-831ef4d0be58 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:05.865372 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1391/2792] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:44:05 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:44:05.866578 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d5a44-e393-455d-9950-78c268525a10 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:05.877747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d5a44-e393-455d-9950-78c268525a10 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:05.882426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1017a3-9c49-4504-b8c7-618d03638d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:05.882975 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1017a3-9c49-4504-b8c7-618d03638d2e admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:05.883254 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1017a3-9c49-4504-b8c7-618d03638d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:05.883549 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1017a3-9c49-4504-b8c7-618d03638d2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:05.884256 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1017a3-9c49-4504-b8c7-618d03638d2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:05.885869 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1017a3-9c49-4504-b8c7-618d03638d2e admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:05.896320 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1402/2793] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:44:05.900327 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cbf54-1468-4c7b-aa6a-831ef4d0be58 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:05.911008 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cbf54-1468-4c7b-aa6a-831ef4d0be58 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:05.916030 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fe111-6c7e-415a-85d0-19747fdc82db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:05.916737 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fe111-6c7e-415a-85d0-19747fdc82db admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:05.916952 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fe111-6c7e-415a-85d0-19747fdc82db admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:05.917235 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fe111-6c7e-415a-85d0-19747fdc82db admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:05.917982 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408fe111-6c7e-415a-85d0-19747fdc82db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:05.920250 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408fe111-6c7e-415a-85d0-19747fdc82db admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:05.932920 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1392/2794] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:44:05.936764 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1017a3-9c49-4504-b8c7-618d03638d2e admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:05.948356 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1017a3-9c49-4504-b8c7-618d03638d2e admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:05.954240 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f36d3-c408-4056-93e6-d293a5fa761b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:05.955192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f36d3-c408-4056-93e6-d293a5fa761b admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:05.955474 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f36d3-c408-4056-93e6-d293a5fa761b admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:05.955750 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f36d3-c408-4056-93e6-d293a5fa761b admin admin] PATH_INFO: `/v3/projects/36afce94f7ab48ecbd123a6859491830/users/0f8e98dd51d9432c8b491d8ab43fd9ff/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:05.973915 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f36d3-c408-4056-93e6-d293a5fa761b admin admin] RBAC: Authorizing `identity:create_grant(project_id=36afce94f7ab48ecbd123a6859491830, user_id=0f8e98dd51d9432c8b491d8ab43fd9ff, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:05.976868 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f36d3-c408-4056-93e6-d293a5fa761b admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:05.993664 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1403/2795] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:44:05 2022] PUT /identity/v3/projects/36afce94f7ab48ecbd123a6859491830/users/0f8e98dd51d9432c8b491d8ab43fd9ff/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:44:06.004778 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7128c4-b146-4734-ae6a-080cf151885c None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:06.005134 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7128c4-b146-4734-ae6a-080cf151885c None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:06.005439 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7128c4-b146-4734-ae6a-080cf151885c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:06.033613 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-7e7128c4-b146-4734-ae6a-080cf151885c None None] MFA Rules not processed for user `0f8e98dd51d9432c8b491d8ab43fd9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:06.034833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-7e7128c4-b146-4734-ae6a-080cf151885c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:06.101269 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1393/2796] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:06.113810 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f36d3-c408-4056-93e6-d293a5fa761b admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:06.116188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-5c7f36d3-c408-4056-93e6-d293a5fa761b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:06.181920 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f36d3-c408-4056-93e6-d293a5fa761b admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:06.186095 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6d0f3-e73f-4614-94c9-2462ae6f876b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:06.187101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6d0f3-e73f-4614-94c9-2462ae6f876b service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:06.187101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6d0f3-e73f-4614-94c9-2462ae6f876b service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:06.187284 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6d0f3-e73f-4614-94c9-2462ae6f876b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:06.201586 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6d0f3-e73f-4614-94c9-2462ae6f876b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:06.203260 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6d0f3-e73f-4614-94c9-2462ae6f876b service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:06.213042 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1404/2797] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:44:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:44:06.750703 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7128c4-b146-4734-ae6a-080cf151885c None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:06.752194 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-7e7128c4-b146-4734-ae6a-080cf151885c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:06.786048 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7128c4-b146-4734-ae6a-080cf151885c None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:06.789682 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e0cbe-f5a3-4e0a-a728-899c17a6f2d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:06.790302 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e0cbe-f5a3-4e0a-a728-899c17a6f2d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:06.790586 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e0cbe-f5a3-4e0a-a728-899c17a6f2d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:06.790870 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e0cbe-f5a3-4e0a-a728-899c17a6f2d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:06.792613 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-7e8e0cbe-f5a3-4e0a-a728-899c17a6f2d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:06.831735 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:06.852353 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8e0cbe-f5a3-4e0a-a728-899c17a6f2d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:06.853823 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8e0cbe-f5a3-4e0a-a728-899c17a6f2d0 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:06.862604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1394/2798] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:44:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:44:07.372917 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6d0f3-e73f-4614-94c9-2462ae6f876b service cinder] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:07.390828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6d0f3-e73f-4614-94c9-2462ae6f876b service cinder] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:07.394324 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d81faa-ed91-41eb-8685-0310862ad956 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:07.395005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d81faa-ed91-41eb-8685-0310862ad956 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:07.395283 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d81faa-ed91-41eb-8685-0310862ad956 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:07.395553 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d81faa-ed91-41eb-8685-0310862ad956 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:07.397324 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-00d81faa-ed91-41eb-8685-0310862ad956 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:07.459630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d81faa-ed91-41eb-8685-0310862ad956 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:07.461142 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d81faa-ed91-41eb-8685-0310862ad956 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:07.465650 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:07.473676 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1405/2799] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:44:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:44:08.007857 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e0cbe-f5a3-4e0a-a728-899c17a6f2d0 service cinder] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:08.009338 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-7e8e0cbe-f5a3-4e0a-a728-899c17a6f2d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:08.066478 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e0cbe-f5a3-4e0a-a728-899c17a6f2d0 service cinder] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:08.070087 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871cf03-c1a8-4d20-bd53-84cc85c9d373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:08.070703 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871cf03-c1a8-4d20-bd53-84cc85c9d373 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:08.070984 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871cf03-c1a8-4d20-bd53-84cc85c9d373 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:08.071263 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871cf03-c1a8-4d20-bd53-84cc85c9d373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:08.072988 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-8871cf03-c1a8-4d20-bd53-84cc85c9d373 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:08.137888 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871cf03-c1a8-4d20-bd53-84cc85c9d373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:08.139416 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871cf03-c1a8-4d20-bd53-84cc85c9d373 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:08.149162 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1395/2800] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:44:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:44:13.434222 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d81faa-ed91-41eb-8685-0310862ad956 service cinder] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:13.500387 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d81faa-ed91-41eb-8685-0310862ad956 service cinder] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:13.505229 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a323f8-39f9-4543-b661-93edebbc73a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:13.505875 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a323f8-39f9-4543-b661-93edebbc73a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:13.506160 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a323f8-39f9-4543-b661-93edebbc73a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:13.506443 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a323f8-39f9-4543-b661-93edebbc73a5 admin admin] PATH_INFO: `/v3/users/0f8e98dd51d9432c8b491d8ab43fd9ff` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:13.507643 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a323f8-39f9-4543-b661-93edebbc73a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f8e98dd51d9432c8b491d8ab43fd9ff)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:13.509635 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a323f8-39f9-4543-b661-93edebbc73a5 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:13.519705 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871cf03-c1a8-4d20-bd53-84cc85c9d373 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:13.521121 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-8871cf03-c1a8-4d20-bd53-84cc85c9d373 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:13.534934 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871cf03-c1a8-4d20-bd53-84cc85c9d373 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:13.539974 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47affc2-bcb1-4208-bbfd-ec3085ee89cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:13.540688 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47affc2-bcb1-4208-bbfd-ec3085ee89cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:13.541230 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47affc2-bcb1-4208-bbfd-ec3085ee89cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:13.541618 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47affc2-bcb1-4208-bbfd-ec3085ee89cf admin admin] PATH_INFO: `/v3/users/7a5de1a4678a42ca8d716ad084e972a2` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:13.542950 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47affc2-bcb1-4208-bbfd-ec3085ee89cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a5de1a4678a42ca8d716ad084e972a2)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:13.545752 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47affc2-bcb1-4208-bbfd-ec3085ee89cf admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:13.557428 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-37a323f8-39f9-4543-b661-93edebbc73a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f8e98dd51d9432c8b491d8ab43fd9ff'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:13.566080 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-37a323f8-39f9-4543-b661-93edebbc73a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f8e98dd51d9432c8b491d8ab43fd9ff'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:13.580002 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-37a323f8-39f9-4543-b661-93edebbc73a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f8e98dd51d9432c8b491d8ab43fd9ff'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:13.587753 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-37a323f8-39f9-4543-b661-93edebbc73a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f8e98dd51d9432c8b491d8ab43fd9ff'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:13.588590 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-37a323f8-39f9-4543-b661-93edebbc73a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f8e98dd51d9432c8b491d8ab43fd9ff'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:13.594908 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1406/2801] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:44:13 2022] DELETE /identity/v3/users/0f8e98dd51d9432c8b491d8ab43fd9ff => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:44:13.597057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a47affc2-bcb1-4208-bbfd-ec3085ee89cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a5de1a4678a42ca8d716ad084e972a2'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:13.601469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe43ca-28f6-4f7f-a9b9-faa6ebe15208 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:13.601948 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe43ca-28f6-4f7f-a9b9-faa6ebe15208 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:13.602288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe43ca-28f6-4f7f-a9b9-faa6ebe15208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:13.605409 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a47affc2-bcb1-4208-bbfd-ec3085ee89cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a5de1a4678a42ca8d716ad084e972a2'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:13.612677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a47affc2-bcb1-4208-bbfd-ec3085ee89cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a5de1a4678a42ca8d716ad084e972a2'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:13.622015 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-a6fe43ca-28f6-4f7f-a9b9-faa6ebe15208 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:13.623218 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-a6fe43ca-28f6-4f7f-a9b9-faa6ebe15208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:13.625130 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a47affc2-bcb1-4208-bbfd-ec3085ee89cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a5de1a4678a42ca8d716ad084e972a2'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:13.625790 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-a47affc2-bcb1-4208-bbfd-ec3085ee89cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a5de1a4678a42ca8d716ad084e972a2'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:13.627855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:13.635970 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1396/2802] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:44:13 2022] DELETE /identity/v3/users/7a5de1a4678a42ca8d716ad084e972a2 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:44:13.637715 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe7f08-9cc7-4cb3-8a35-96b8db62ada8 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:13.638276 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe7f08-9cc7-4cb3-8a35-96b8db62ada8 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:13.638793 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe7f08-9cc7-4cb3-8a35-96b8db62ada8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:13.658494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-befe7f08-9cc7-4cb3-8a35-96b8db62ada8 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:13.659738 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-befe7f08-9cc7-4cb3-8a35-96b8db62ada8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:13.686773 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1407/2803] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:44:13.696515 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe43ca-28f6-4f7f-a9b9-faa6ebe15208 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:13.697922 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-a6fe43ca-28f6-4f7f-a9b9-faa6ebe15208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:13.720645 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:13.721368 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1397/2804] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:44:13.731078 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe7f08-9cc7-4cb3-8a35-96b8db62ada8 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:13.732460 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-befe7f08-9cc7-4cb3-8a35-96b8db62ada8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:13.763574 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe43ca-28f6-4f7f-a9b9-faa6ebe15208 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:13.789712 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe7f08-9cc7-4cb3-8a35-96b8db62ada8 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:13.793896 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c3ddc-15ee-4cf2-b0b9-ec934588b037 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:13.794515 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c3ddc-15ee-4cf2-b0b9-ec934588b037 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:13.794799 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c3ddc-15ee-4cf2-b0b9-ec934588b037 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:13.795077 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c3ddc-15ee-4cf2-b0b9-ec934588b037 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:13.804208 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c3ddc-15ee-4cf2-b0b9-ec934588b037 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:13.805677 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c3ddc-15ee-4cf2-b0b9-ec934588b037 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:13.814112 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72abec7e-f6c1-4496-b71b-50159649c20a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:13.814726 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abec7e-f6c1-4496-b71b-50159649c20a service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:13.815003 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abec7e-f6c1-4496-b71b-50159649c20a service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:13.815273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abec7e-f6c1-4496-b71b-50159649c20a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:13.816190 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1398/2805] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:44:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:44:13.824791 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72abec7e-f6c1-4496-b71b-50159649c20a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:13.826287 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72abec7e-f6c1-4496-b71b-50159649c20a service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:13.836005 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1408/2806] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:44:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:44:14.105134 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c3ddc-15ee-4cf2-b0b9-ec934588b037 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:14.107151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-713c3ddc-15ee-4cf2-b0b9-ec934588b037 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:14.111379 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72abec7e-f6c1-4496-b71b-50159649c20a service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:14.113077 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-72abec7e-f6c1-4496-b71b-50159649c20a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:14.126181 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c3ddc-15ee-4cf2-b0b9-ec934588b037 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:14.131393 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d806440-1542-42f7-9f75-52852e09899f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:14.132043 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d806440-1542-42f7-9f75-52852e09899f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:14.132374 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d806440-1542-42f7-9f75-52852e09899f admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:14.132656 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d806440-1542-42f7-9f75-52852e09899f admin admin] PATH_INFO: `/v3/projects/f4291557abaf483d8cbfbbc26e14eb31` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:14.135563 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d806440-1542-42f7-9f75-52852e09899f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4291557abaf483d8cbfbbc26e14eb31)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:14.135851 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72abec7e-f6c1-4496-b71b-50159649c20a service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:14.138005 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d806440-1542-42f7-9f75-52852e09899f admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:14.144749 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ef13a-9d75-4560-85e1-d89152173ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:14.145979 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ef13a-9d75-4560-85e1-d89152173ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:14.146140 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ef13a-9d75-4560-85e1-d89152173ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:14.146465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ef13a-9d75-4560-85e1-d89152173ef8 admin admin] PATH_INFO: `/v3/projects/36afce94f7ab48ecbd123a6859491830` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:14.147987 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ef13a-9d75-4560-85e1-d89152173ef8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36afce94f7ab48ecbd123a6859491830)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:14.151608 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ef13a-9d75-4560-85e1-d89152173ef8 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:14.204093 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-5d806440-1542-42f7-9f75-52852e09899f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4291557abaf483d8cbfbbc26e14eb31', 'request_id': 'req-5d806440-1542-42f7-9f75-52852e09899f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:14.210819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-5d806440-1542-42f7-9f75-52852e09899f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4291557abaf483d8cbfbbc26e14eb31', 'request_id': 'req-5d806440-1542-42f7-9f75-52852e09899f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:14.220081 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-5d806440-1542-42f7-9f75-52852e09899f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4291557abaf483d8cbfbbc26e14eb31', 'request_id': 'req-5d806440-1542-42f7-9f75-52852e09899f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:14.228856 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-810ef13a-9d75-4560-85e1-d89152173ef8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36afce94f7ab48ecbd123a6859491830', 'request_id': 'req-810ef13a-9d75-4560-85e1-d89152173ef8'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:14.234840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-5d806440-1542-42f7-9f75-52852e09899f admin admin] The token cache is being invalidate because project f4291557abaf483d8cbfbbc26e14eb31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:44:14.235236 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-5d806440-1542-42f7-9f75-52852e09899f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:14.235936 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-5d806440-1542-42f7-9f75-52852e09899f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:14.237802 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1399/2807] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:44:14 2022] DELETE /identity/v3/projects/f4291557abaf483d8cbfbbc26e14eb31 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:44:14.237946 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-810ef13a-9d75-4560-85e1-d89152173ef8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36afce94f7ab48ecbd123a6859491830', 'request_id': 'req-810ef13a-9d75-4560-85e1-d89152173ef8'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:14.241463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d806440-1542-42f7-9f75-52852e09899f admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:14.242980 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-5d806440-1542-42f7-9f75-52852e09899f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:14.244805 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-810ef13a-9d75-4560-85e1-d89152173ef8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36afce94f7ab48ecbd123a6859491830', 'request_id': 'req-810ef13a-9d75-4560-85e1-d89152173ef8'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:14.258073 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-810ef13a-9d75-4560-85e1-d89152173ef8 admin admin] The token cache is being invalidate because project 36afce94f7ab48ecbd123a6859491830 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:44:14.258367 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-810ef13a-9d75-4560-85e1-d89152173ef8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:14.259017 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-810ef13a-9d75-4560-85e1-d89152173ef8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:14.260526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1409/2808] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:44:14 2022] DELETE /identity/v3/projects/36afce94f7ab48ecbd123a6859491830 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:44:14.290635 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:14.356848 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d806440-1542-42f7-9f75-52852e09899f admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:14.426521 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63115d26-f735-4e54-9da7-e46f76e53060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:14.427376 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63115d26-f735-4e54-9da7-e46f76e53060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:14.427739 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63115d26-f735-4e54-9da7-e46f76e53060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:14.428195 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63115d26-f735-4e54-9da7-e46f76e53060 admin admin] PATH_INFO: `/v3/users/dae35e04bf3941b38829b35ef9a31a2f` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:14.429976 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63115d26-f735-4e54-9da7-e46f76e53060 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dae35e04bf3941b38829b35ef9a31a2f)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:14.432053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63115d26-f735-4e54-9da7-e46f76e53060 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:14.491846 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-63115d26-f735-4e54-9da7-e46f76e53060 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dae35e04bf3941b38829b35ef9a31a2f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:14.504748 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-63115d26-f735-4e54-9da7-e46f76e53060 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dae35e04bf3941b38829b35ef9a31a2f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:14.516764 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-63115d26-f735-4e54-9da7-e46f76e53060 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dae35e04bf3941b38829b35ef9a31a2f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:14.532297 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-63115d26-f735-4e54-9da7-e46f76e53060 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dae35e04bf3941b38829b35ef9a31a2f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:14.533623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-63115d26-f735-4e54-9da7-e46f76e53060 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dae35e04bf3941b38829b35ef9a31a2f'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:14.544067 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1400/2809] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:44:14 2022] DELETE /identity/v3/users/dae35e04bf3941b38829b35ef9a31a2f => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:44:14.842505 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ef13a-9d75-4560-85e1-d89152173ef8 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:14.844070 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-810ef13a-9d75-4560-85e1-d89152173ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:14.861188 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ef13a-9d75-4560-85e1-d89152173ef8 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:14.928018 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd9053-172d-41ca-9952-d0bb8f9d3e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:14.928675 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd9053-172d-41ca-9952-d0bb8f9d3e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:14.928993 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd9053-172d-41ca-9952-d0bb8f9d3e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:14.929271 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd9053-172d-41ca-9952-d0bb8f9d3e75 admin admin] PATH_INFO: `/v3/projects/6d8d59d2b433466b897ce8ecd37df11b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:14.930436 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cd9053-172d-41ca-9952-d0bb8f9d3e75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d8d59d2b433466b897ce8ecd37df11b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:14.932386 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cd9053-172d-41ca-9952-d0bb8f9d3e75 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:14.933946 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:14.983606 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-90cd9053-172d-41ca-9952-d0bb8f9d3e75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d8d59d2b433466b897ce8ecd37df11b', 'request_id': 'req-90cd9053-172d-41ca-9952-d0bb8f9d3e75'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:14.991684 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-90cd9053-172d-41ca-9952-d0bb8f9d3e75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d8d59d2b433466b897ce8ecd37df11b', 'request_id': 'req-90cd9053-172d-41ca-9952-d0bb8f9d3e75'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:14.997651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-90cd9053-172d-41ca-9952-d0bb8f9d3e75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d8d59d2b433466b897ce8ecd37df11b', 'request_id': 'req-90cd9053-172d-41ca-9952-d0bb8f9d3e75'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:15.009276 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-90cd9053-172d-41ca-9952-d0bb8f9d3e75 admin admin] The token cache is being invalidate because project 6d8d59d2b433466b897ce8ecd37df11b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:44:15.009569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-90cd9053-172d-41ca-9952-d0bb8f9d3e75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:15.010249 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-90cd9053-172d-41ca-9952-d0bb8f9d3e75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:15.011809 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1410/2810] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:44:14 2022] DELETE /identity/v3/projects/6d8d59d2b433466b897ce8ecd37df11b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:44:15.016049 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63115d26-f735-4e54-9da7-e46f76e53060 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:15.017495 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-63115d26-f735-4e54-9da7-e46f76e53060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:15.032296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63115d26-f735-4e54-9da7-e46f76e53060 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:15.084391 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:15.097060 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e599e42-08ca-466c-9b91-af7c4bc10677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:15.098462 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e599e42-08ca-466c-9b91-af7c4bc10677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:15.099733 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e599e42-08ca-466c-9b91-af7c4bc10677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:15.100365 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e599e42-08ca-466c-9b91-af7c4bc10677 admin admin] PATH_INFO: `/v3/users/f85d84dbe6b64ac8b809c4f0cb19bf20` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:15.102544 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e599e42-08ca-466c-9b91-af7c4bc10677 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f85d84dbe6b64ac8b809c4f0cb19bf20)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:15.107523 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e599e42-08ca-466c-9b91-af7c4bc10677 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:15.178188 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8e599e42-08ca-466c-9b91-af7c4bc10677 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f85d84dbe6b64ac8b809c4f0cb19bf20'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:15.188817 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8e599e42-08ca-466c-9b91-af7c4bc10677 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f85d84dbe6b64ac8b809c4f0cb19bf20'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:15.196401 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8e599e42-08ca-466c-9b91-af7c4bc10677 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f85d84dbe6b64ac8b809c4f0cb19bf20'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:15.209378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8e599e42-08ca-466c-9b91-af7c4bc10677 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f85d84dbe6b64ac8b809c4f0cb19bf20'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:15.210076 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-8e599e42-08ca-466c-9b91-af7c4bc10677 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f85d84dbe6b64ac8b809c4f0cb19bf20'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:15.216515 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1401/2811] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:44:15 2022] DELETE /identity/v3/users/f85d84dbe6b64ac8b809c4f0cb19bf20 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:44:15.495761 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd9053-172d-41ca-9952-d0bb8f9d3e75 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:15.497380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-90cd9053-172d-41ca-9952-d0bb8f9d3e75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:15.513080 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd9053-172d-41ca-9952-d0bb8f9d3e75 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:15.576776 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753da86-c727-48f7-9a61-fb3512c91272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:15.577463 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753da86-c727-48f7-9a61-fb3512c91272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:15.577749 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753da86-c727-48f7-9a61-fb3512c91272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:15.578048 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753da86-c727-48f7-9a61-fb3512c91272 admin admin] PATH_INFO: `/v3/projects/7b085ad470b2405e93c9606ba2655c23` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:15.579209 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6753da86-c727-48f7-9a61-fb3512c91272 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b085ad470b2405e93c9606ba2655c23)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:15.581196 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6753da86-c727-48f7-9a61-fb3512c91272 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:15.630779 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6753da86-c727-48f7-9a61-fb3512c91272 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b085ad470b2405e93c9606ba2655c23', 'request_id': 'req-6753da86-c727-48f7-9a61-fb3512c91272'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:15.639814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6753da86-c727-48f7-9a61-fb3512c91272 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b085ad470b2405e93c9606ba2655c23', 'request_id': 'req-6753da86-c727-48f7-9a61-fb3512c91272'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:15.645845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6753da86-c727-48f7-9a61-fb3512c91272 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b085ad470b2405e93c9606ba2655c23', 'request_id': 'req-6753da86-c727-48f7-9a61-fb3512c91272'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:15.657904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6753da86-c727-48f7-9a61-fb3512c91272 admin admin] The token cache is being invalidate because project 7b085ad470b2405e93c9606ba2655c23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:44:15.658291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6753da86-c727-48f7-9a61-fb3512c91272 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:15.658916 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-6753da86-c727-48f7-9a61-fb3512c91272 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:15.660510 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1411/2812] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:44:15 2022] DELETE /identity/v3/projects/7b085ad470b2405e93c9606ba2655c23 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:44:15.665439 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bca8c0-33ab-493d-b0a4-4fda587e3e51 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:15.665966 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bca8c0-33ab-493d-b0a4-4fda587e3e51 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:15.666071 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bca8c0-33ab-493d-b0a4-4fda587e3e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:15.690600 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-63bca8c0-33ab-493d-b0a4-4fda587e3e51 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:15.691198 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753da86-c727-48f7-9a61-fb3512c91272 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:15.691833 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-63bca8c0-33ab-493d-b0a4-4fda587e3e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:15.692883 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-6753da86-c727-48f7-9a61-fb3512c91272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:15.720498 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:15.758625 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1402/2813] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:15.766318 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32848f56-bb0d-4bb5-9c49-2b3fef164f8b None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:15.766564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753da86-c727-48f7-9a61-fb3512c91272 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:15.766699 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32848f56-bb0d-4bb5-9c49-2b3fef164f8b None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:15.766902 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32848f56-bb0d-4bb5-9c49-2b3fef164f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:15.771375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af460ded-aeb8-4c83-86db-3d71fd08b1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:15.772052 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af460ded-aeb8-4c83-86db-3d71fd08b1af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:15.772338 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af460ded-aeb8-4c83-86db-3d71fd08b1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:15.772622 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af460ded-aeb8-4c83-86db-3d71fd08b1af admin admin] PATH_INFO: `/v3/users/8b5a532003384e5180c73124c7b62077` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:15.773821 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af460ded-aeb8-4c83-86db-3d71fd08b1af admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b5a532003384e5180c73124c7b62077)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:15.775742 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af460ded-aeb8-4c83-86db-3d71fd08b1af admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:15.787368 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-32848f56-bb0d-4bb5-9c49-2b3fef164f8b None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:15.788592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-32848f56-bb0d-4bb5-9c49-2b3fef164f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:15.796795 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1403/2814] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:44:15.800852 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32848f56-bb0d-4bb5-9c49-2b3fef164f8b None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:15.811523 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32848f56-bb0d-4bb5-9c49-2b3fef164f8b None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:15.816808 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f5b0f-ce0c-4ccd-8464-52af079b517b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:15.817486 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f5b0f-ce0c-4ccd-8464-52af079b517b admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:15.817837 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f5b0f-ce0c-4ccd-8464-52af079b517b admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:15.818051 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f5b0f-ce0c-4ccd-8464-52af079b517b admin admin] PATH_INFO: `/v3/domains` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:15.818742 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f5b0f-ce0c-4ccd-8464-52af079b517b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:44:15.819052 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f5b0f-ce0c-4ccd-8464-52af079b517b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:15.820690 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f5b0f-ce0c-4ccd-8464-52af079b517b admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:15.828774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-af460ded-aeb8-4c83-86db-3d71fd08b1af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b5a532003384e5180c73124c7b62077'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:15.834860 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1404/2815] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:44:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:44:15.839604 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f5b0f-ce0c-4ccd-8464-52af079b517b admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:15.841259 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-af460ded-aeb8-4c83-86db-3d71fd08b1af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b5a532003384e5180c73124c7b62077'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:15.850291 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f5b0f-ce0c-4ccd-8464-52af079b517b admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:15.854192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-af460ded-aeb8-4c83-86db-3d71fd08b1af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b5a532003384e5180c73124c7b62077'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:15.855267 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ce941-20e3-49fa-a751-9d91723184a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:15.856538 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ce941-20e3-49fa-a751-9d91723184a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:15.856821 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ce941-20e3-49fa-a751-9d91723184a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:15.857131 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ce941-20e3-49fa-a751-9d91723184a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:15.857799 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ce941-20e3-49fa-a751-9d91723184a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:15.859693 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ce941-20e3-49fa-a751-9d91723184a5 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:15.861843 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-af460ded-aeb8-4c83-86db-3d71fd08b1af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b5a532003384e5180c73124c7b62077'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:15.862549 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-af460ded-aeb8-4c83-86db-3d71fd08b1af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b5a532003384e5180c73124c7b62077'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:15.868566 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1412/2816] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:44:15 2022] DELETE /identity/v3/users/8b5a532003384e5180c73124c7b62077 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:44:15.889498 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1405/2817] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:44:15 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:44:15.891026 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af460ded-aeb8-4c83-86db-3d71fd08b1af admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:15.892449 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-af460ded-aeb8-4c83-86db-3d71fd08b1af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:15.957717 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af460ded-aeb8-4c83-86db-3d71fd08b1af admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:16.000610 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:16.015727 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856a23f-8b39-42ba-b3ac-aa9a9a546a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:16.016349 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856a23f-8b39-42ba-b3ac-aa9a9a546a5a admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:16.016632 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856a23f-8b39-42ba-b3ac-aa9a9a546a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:16.016933 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856a23f-8b39-42ba-b3ac-aa9a9a546a5a admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:16.017611 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856a23f-8b39-42ba-b3ac-aa9a9a546a5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:16.019555 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856a23f-8b39-42ba-b3ac-aa9a9a546a5a admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:16.041606 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1413/2818] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:44:15 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:44:16.046209 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ce941-20e3-49fa-a751-9d91723184a5 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:16.047274 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856a23f-8b39-42ba-b3ac-aa9a9a546a5a admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:16.048196 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-746ce941-20e3-49fa-a751-9d91723184a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:16.059938 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856a23f-8b39-42ba-b3ac-aa9a9a546a5a admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:16.064732 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45941b9-f122-49e2-b71c-684e528093f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:16.065388 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45941b9-f122-49e2-b71c-684e528093f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:16.065674 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45941b9-f122-49e2-b71c-684e528093f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:16.065952 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45941b9-f122-49e2-b71c-684e528093f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:16.066650 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45941b9-f122-49e2-b71c-684e528093f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:16.068978 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45941b9-f122-49e2-b71c-684e528093f0 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:16.070791 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ce941-20e3-49fa-a751-9d91723184a5 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:16.076422 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58245c-f1d1-4017-b8d0-59efe0e66180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:16.077097 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58245c-f1d1-4017-b8d0-59efe0e66180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:16.077387 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58245c-f1d1-4017-b8d0-59efe0e66180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:16.077664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58245c-f1d1-4017-b8d0-59efe0e66180 admin admin] PATH_INFO: `/v3/projects/8ef15b981e8d4885941afecc5cc74d1a` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:16.078841 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58245c-f1d1-4017-b8d0-59efe0e66180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ef15b981e8d4885941afecc5cc74d1a)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:16.080640 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1414/2819] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:44:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:44:16.083845 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58245c-f1d1-4017-b8d0-59efe0e66180 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:16.084834 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45941b9-f122-49e2-b71c-684e528093f0 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:16.095725 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45941b9-f122-49e2-b71c-684e528093f0 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:16.101244 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e351652-3210-4400-8aec-3f9bccefa574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:16.101842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e351652-3210-4400-8aec-3f9bccefa574 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:16.102123 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e351652-3210-4400-8aec-3f9bccefa574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:16.102396 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e351652-3210-4400-8aec-3f9bccefa574 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:16.103090 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e351652-3210-4400-8aec-3f9bccefa574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:16.104672 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e351652-3210-4400-8aec-3f9bccefa574 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:16.113053 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:16.117023 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1415/2820] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:44:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:44:16.121522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e351652-3210-4400-8aec-3f9bccefa574 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:16.130929 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0a58245c-f1d1-4017-b8d0-59efe0e66180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ef15b981e8d4885941afecc5cc74d1a', 'request_id': 'req-0a58245c-f1d1-4017-b8d0-59efe0e66180'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:16.132342 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e351652-3210-4400-8aec-3f9bccefa574 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:16.137132 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0a58245c-f1d1-4017-b8d0-59efe0e66180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ef15b981e8d4885941afecc5cc74d1a', 'request_id': 'req-0a58245c-f1d1-4017-b8d0-59efe0e66180'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:16.144722 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0a58245c-f1d1-4017-b8d0-59efe0e66180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ef15b981e8d4885941afecc5cc74d1a', 'request_id': 'req-0a58245c-f1d1-4017-b8d0-59efe0e66180'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:16.160742 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0a58245c-f1d1-4017-b8d0-59efe0e66180 admin admin] The token cache is being invalidate because project 8ef15b981e8d4885941afecc5cc74d1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:44:16.161273 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0a58245c-f1d1-4017-b8d0-59efe0e66180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:16.161959 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-0a58245c-f1d1-4017-b8d0-59efe0e66180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:16.163537 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1406/2821] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:44:16 2022] DELETE /identity/v3/projects/8ef15b981e8d4885941afecc5cc74d1a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:44:16.167796 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58245c-f1d1-4017-b8d0-59efe0e66180 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:16.169261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-0a58245c-f1d1-4017-b8d0-59efe0e66180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:16.193640 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f7250-906a-4bd6-8496-a58aaafc46b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:16.194307 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f7250-906a-4bd6-8496-a58aaafc46b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:16.194583 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f7250-906a-4bd6-8496-a58aaafc46b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:16.194855 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f7250-906a-4bd6-8496-a58aaafc46b9 admin admin] PATH_INFO: `/v3/projects/75eb3f6360074168b3382e36594694f4/users/b3cd6c70efe1435ab0d753500d9ab302/roles/8de48592893c43789e1d40158943fd52` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:16.209786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f7250-906a-4bd6-8496-a58aaafc46b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75eb3f6360074168b3382e36594694f4, user_id=b3cd6c70efe1435ab0d753500d9ab302, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:16.212541 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f7250-906a-4bd6-8496-a58aaafc46b9 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:16.226723 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1416/2822] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:44:16 2022] PUT /identity/v3/projects/75eb3f6360074168b3382e36594694f4/users/b3cd6c70efe1435ab0d753500d9ab302/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:44:16.237119 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e835e-4222-4f22-b980-d6020a0b73e1 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:16.237383 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e835e-4222-4f22-b980-d6020a0b73e1 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:16.237651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e835e-4222-4f22-b980-d6020a0b73e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:16.244583 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58245c-f1d1-4017-b8d0-59efe0e66180 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:16.264199 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-e04e835e-4222-4f22-b980-d6020a0b73e1 None None] MFA Rules not processed for user `b3cd6c70efe1435ab0d753500d9ab302`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:16.265426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e04e835e-4222-4f22-b980-d6020a0b73e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:16.301719 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d5426-7ac4-49c1-ba43-d93df454ee24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:16.302368 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d5426-7ac4-49c1-ba43-d93df454ee24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:16.302653 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d5426-7ac4-49c1-ba43-d93df454ee24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:16.302936 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d5426-7ac4-49c1-ba43-d93df454ee24 admin admin] PATH_INFO: `/v3/users/5923756dc65442e692e33f52a4bc1f49` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:16.304241 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d5426-7ac4-49c1-ba43-d93df454ee24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5923756dc65442e692e33f52a4bc1f49)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:16.306233 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d5426-7ac4-49c1-ba43-d93df454ee24 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:16.330488 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1417/2823] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:44:16.342425 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e835e-4222-4f22-b980-d6020a0b73e1 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:16.344453 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-e04e835e-4222-4f22-b980-d6020a0b73e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:16.361256 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-940d5426-7ac4-49c1-ba43-d93df454ee24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5923756dc65442e692e33f52a4bc1f49'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:16.368739 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-940d5426-7ac4-49c1-ba43-d93df454ee24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5923756dc65442e692e33f52a4bc1f49'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:16.375104 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:16.377141 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-940d5426-7ac4-49c1-ba43-d93df454ee24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5923756dc65442e692e33f52a4bc1f49'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:16.378262 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:16.392922 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-940d5426-7ac4-49c1-ba43-d93df454ee24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5923756dc65442e692e33f52a4bc1f49'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:16.393732 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-940d5426-7ac4-49c1-ba43-d93df454ee24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5923756dc65442e692e33f52a4bc1f49'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:16.399564 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1407/2824] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:44:16 2022] DELETE /identity/v3/users/5923756dc65442e692e33f52a4bc1f49 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:44:16.408517 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e835e-4222-4f22-b980-d6020a0b73e1 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:16.458840 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe845c8-97aa-42b3-ba77-5bcba026c959 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:16.459456 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe845c8-97aa-42b3-ba77-5bcba026c959 service cinder] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:16.460271 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe845c8-97aa-42b3-ba77-5bcba026c959 service cinder] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:16.460500 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe845c8-97aa-42b3-ba77-5bcba026c959 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:16.462353 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-fbe845c8-97aa-42b3-ba77-5bcba026c959 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:16.522096 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe845c8-97aa-42b3-ba77-5bcba026c959 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:16.523578 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe845c8-97aa-42b3-ba77-5bcba026c959 service cinder] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:16.532735 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1418/2825] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:44:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:44:16.578016 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d5426-7ac4-49c1-ba43-d93df454ee24 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:16.579601 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-940d5426-7ac4-49c1-ba43-d93df454ee24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:16.656111 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d5426-7ac4-49c1-ba43-d93df454ee24 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:16.661235 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b243ed8-f9db-41fe-b574-d33fe9a98ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:16.661880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b243ed8-f9db-41fe-b574-d33fe9a98ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:16.662170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b243ed8-f9db-41fe-b574-d33fe9a98ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:16.662592 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b243ed8-f9db-41fe-b574-d33fe9a98ba9 admin admin] PATH_INFO: `/v3/projects/7e8ea5b970244614987d988e0f9673b7` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:16.664015 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b243ed8-f9db-41fe-b574-d33fe9a98ba9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e8ea5b970244614987d988e0f9673b7)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:16.666827 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b243ed8-f9db-41fe-b574-d33fe9a98ba9 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:16.718815 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-7b243ed8-f9db-41fe-b574-d33fe9a98ba9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e8ea5b970244614987d988e0f9673b7', 'request_id': 'req-7b243ed8-f9db-41fe-b574-d33fe9a98ba9'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:16.725102 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-7b243ed8-f9db-41fe-b574-d33fe9a98ba9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e8ea5b970244614987d988e0f9673b7', 'request_id': 'req-7b243ed8-f9db-41fe-b574-d33fe9a98ba9'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:16.733811 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-7b243ed8-f9db-41fe-b574-d33fe9a98ba9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e8ea5b970244614987d988e0f9673b7', 'request_id': 'req-7b243ed8-f9db-41fe-b574-d33fe9a98ba9'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:16.746861 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-7b243ed8-f9db-41fe-b574-d33fe9a98ba9 admin admin] The token cache is being invalidate because project 7e8ea5b970244614987d988e0f9673b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:44:16.747156 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-7b243ed8-f9db-41fe-b574-d33fe9a98ba9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:16.747810 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-7b243ed8-f9db-41fe-b574-d33fe9a98ba9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:16.749726 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1408/2826] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:44:16 2022] DELETE /identity/v3/projects/7e8ea5b970244614987d988e0f9673b7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:44:16.756229 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9266f1-9203-4ce0-8c92-d79cdf84e78d None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:16.756604 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9266f1-9203-4ce0-8c92-d79cdf84e78d None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:16.756912 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9266f1-9203-4ce0-8c92-d79cdf84e78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:16.779164 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-dc9266f1-9203-4ce0-8c92-d79cdf84e78d None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:16.781248 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-dc9266f1-9203-4ce0-8c92-d79cdf84e78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:16.855764 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1419/2827] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:16.863727 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbb129-0d28-49d7-92f9-ea10f711c565 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:16.864016 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbb129-0d28-49d7-92f9-ea10f711c565 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:16.864292 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bbb129-0d28-49d7-92f9-ea10f711c565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:16.884915 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-d5bbb129-0d28-49d7-92f9-ea10f711c565 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:16.886153 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d5bbb129-0d28-49d7-92f9-ea10f711c565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:16.894158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1409/2828] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:16.898955 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9266f1-9203-4ce0-8c92-d79cdf84e78d None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:16.909305 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9266f1-9203-4ce0-8c92-d79cdf84e78d None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:16.913954 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6437b-ba3d-4d6e-9d66-639731c8eb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:16.914564 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6437b-ba3d-4d6e-9d66-639731c8eb40 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:16.914848 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6437b-ba3d-4d6e-9d66-639731c8eb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:16.915147 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6437b-ba3d-4d6e-9d66-639731c8eb40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:16.915839 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6437b-ba3d-4d6e-9d66-639731c8eb40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:44:16.916144 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6437b-ba3d-4d6e-9d66-639731c8eb40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:16.918281 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6437b-ba3d-4d6e-9d66-639731c8eb40 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:16.929594 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1420/2829] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:44:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 16:44:16.934304 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbb129-0d28-49d7-92f9-ea10f711c565 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:16.943370 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bbb129-0d28-49d7-92f9-ea10f711c565 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:16.948034 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd21326-6c91-445e-9a53-9380f8dffe36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:16.948632 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd21326-6c91-445e-9a53-9380f8dffe36 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:16.948945 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd21326-6c91-445e-9a53-9380f8dffe36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:16.949253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd21326-6c91-445e-9a53-9380f8dffe36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:16.949904 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd21326-6c91-445e-9a53-9380f8dffe36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:16.951794 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd21326-6c91-445e-9a53-9380f8dffe36 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:16.979307 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1410/2830] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:44:16 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:44:16.984040 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6437b-ba3d-4d6e-9d66-639731c8eb40 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:16.994179 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6437b-ba3d-4d6e-9d66-639731c8eb40 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:17.014815 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=100 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:17.053618 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8c8d8-24b6-4756-8779-8e98cf0079a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:17.054231 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8c8d8-24b6-4756-8779-8e98cf0079a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:17.054512 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8c8d8-24b6-4756-8779-8e98cf0079a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:17.054784 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8c8d8-24b6-4756-8779-8e98cf0079a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:17.055544 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8c8d8-24b6-4756-8779-8e98cf0079a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:17.057607 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8c8d8-24b6-4756-8779-8e98cf0079a7 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:17.078800 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1421/2831] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:44:16 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:44:17.083341 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd21326-6c91-445e-9a53-9380f8dffe36 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:17.094442 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd21326-6c91-445e-9a53-9380f8dffe36 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:17.099809 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbef121-bbc3-403e-a86c-bd2f7bfb2227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:17.100425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbef121-bbc3-403e-a86c-bd2f7bfb2227 admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:17.100706 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbef121-bbc3-403e-a86c-bd2f7bfb2227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:17.101006 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbef121-bbc3-403e-a86c-bd2f7bfb2227 admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:17.101694 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbef121-bbc3-403e-a86c-bd2f7bfb2227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:17.103281 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbef121-bbc3-403e-a86c-bd2f7bfb2227 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:17.114520 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1411/2832] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:44:17.118740 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8c8d8-24b6-4756-8779-8e98cf0079a7 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:17.130701 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8c8d8-24b6-4756-8779-8e98cf0079a7 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:17.135962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040ccfd-5c55-4055-96cc-6d44211d5ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:17.136603 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040ccfd-5c55-4055-96cc-6d44211d5ada admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:17.136904 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040ccfd-5c55-4055-96cc-6d44211d5ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:17.137252 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040ccfd-5c55-4055-96cc-6d44211d5ada admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:17.137961 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040ccfd-5c55-4055-96cc-6d44211d5ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:17.139611 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040ccfd-5c55-4055-96cc-6d44211d5ada admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:17.150398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1422/2833] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:44:17.154790 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbef121-bbc3-403e-a86c-bd2f7bfb2227 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:17.165173 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbef121-bbc3-403e-a86c-bd2f7bfb2227 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:17.169892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769dbe73-2d0b-4d3c-bbf7-188caa624617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:17.170553 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769dbe73-2d0b-4d3c-bbf7-188caa624617 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:17.170839 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769dbe73-2d0b-4d3c-bbf7-188caa624617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:17.171115 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769dbe73-2d0b-4d3c-bbf7-188caa624617 admin admin] PATH_INFO: `/v3/projects/347f7dbd4fd0413e8721edee116bf2a2/users/71f2691b1e75408889d57a01c06a6d16/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:17.186070 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769dbe73-2d0b-4d3c-bbf7-188caa624617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=347f7dbd4fd0413e8721edee116bf2a2, user_id=71f2691b1e75408889d57a01c06a6d16, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:17.188731 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769dbe73-2d0b-4d3c-bbf7-188caa624617 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:17.202105 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1412/2834] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:44:17 2022] PUT /identity/v3/projects/347f7dbd4fd0413e8721edee116bf2a2/users/71f2691b1e75408889d57a01c06a6d16/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:44:17.213398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05ee2e-1160-4bd4-b843-320a89a3a5e3 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:17.213735 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05ee2e-1160-4bd4-b843-320a89a3a5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:17.214017 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05ee2e-1160-4bd4-b843-320a89a3a5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:17.241087 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-2f05ee2e-1160-4bd4-b843-320a89a3a5e3 None None] MFA Rules not processed for user `71f2691b1e75408889d57a01c06a6d16`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:17.242227 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-2f05ee2e-1160-4bd4-b843-320a89a3a5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:17.305184 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1423/2835] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:17 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:17.317057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769dbe73-2d0b-4d3c-bbf7-188caa624617 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:17.318752 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-769dbe73-2d0b-4d3c-bbf7-188caa624617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:17.329191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:17.378475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769dbe73-2d0b-4d3c-bbf7-188caa624617 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:17.382151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d221e-4ff6-4506-bb82-31b41f16dd0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:17.382769 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d221e-4ff6-4506-bb82-31b41f16dd0f service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:17.383057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d221e-4ff6-4506-bb82-31b41f16dd0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:17.383336 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d221e-4ff6-4506-bb82-31b41f16dd0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:17.392301 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d221e-4ff6-4506-bb82-31b41f16dd0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:17.393794 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d221e-4ff6-4506-bb82-31b41f16dd0f service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:17.402601 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1413/2836] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:44:17 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:44:19.107588 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05ee2e-1160-4bd4-b843-320a89a3a5e3 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:19.109197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-2f05ee2e-1160-4bd4-b843-320a89a3a5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:19.185521 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05ee2e-1160-4bd4-b843-320a89a3a5e3 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:19.190358 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d167799-08c4-4a1b-b6d0-24f4ba484a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:19.191000 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d167799-08c4-4a1b-b6d0-24f4ba484a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:19.192800 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d167799-08c4-4a1b-b6d0-24f4ba484a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:19.192800 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d167799-08c4-4a1b-b6d0-24f4ba484a43 admin admin] PATH_INFO: `/v3/users/b3cd6c70efe1435ab0d753500d9ab302` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:19.193529 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d167799-08c4-4a1b-b6d0-24f4ba484a43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3cd6c70efe1435ab0d753500d9ab302)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:19.195581 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d167799-08c4-4a1b-b6d0-24f4ba484a43 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:19.244203 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:19.256321 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-2d167799-08c4-4a1b-b6d0-24f4ba484a43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3cd6c70efe1435ab0d753500d9ab302'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:19.265559 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-2d167799-08c4-4a1b-b6d0-24f4ba484a43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3cd6c70efe1435ab0d753500d9ab302'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:19.278489 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-2d167799-08c4-4a1b-b6d0-24f4ba484a43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3cd6c70efe1435ab0d753500d9ab302'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:19.285526 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-2d167799-08c4-4a1b-b6d0-24f4ba484a43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3cd6c70efe1435ab0d753500d9ab302'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:19.286197 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-2d167799-08c4-4a1b-b6d0-24f4ba484a43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3cd6c70efe1435ab0d753500d9ab302'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:19.291960 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1424/2837] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:44:19 2022] DELETE /identity/v3/users/b3cd6c70efe1435ab0d753500d9ab302 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:44:19.296537 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fd27c-70c3-4a52-a0da-fffed19fc003 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:19.297056 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fd27c-70c3-4a52-a0da-fffed19fc003 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:19.297174 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fd27c-70c3-4a52-a0da-fffed19fc003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:19.315544 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-905fd27c-70c3-4a52-a0da-fffed19fc003 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:19.316836 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-905fd27c-70c3-4a52-a0da-fffed19fc003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:19.375464 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1414/2838] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:19.385280 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d167799-08c4-4a1b-b6d0-24f4ba484a43 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:19.387336 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-2d167799-08c4-4a1b-b6d0-24f4ba484a43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:19.442862 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d167799-08c4-4a1b-b6d0-24f4ba484a43 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:19.446378 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f599a-9ece-4888-9be6-8b96df456cc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:19.446980 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f599a-9ece-4888-9be6-8b96df456cc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:19.447258 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f599a-9ece-4888-9be6-8b96df456cc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:19.447535 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f599a-9ece-4888-9be6-8b96df456cc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:19.458322 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f599a-9ece-4888-9be6-8b96df456cc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:19.459761 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f599a-9ece-4888-9be6-8b96df456cc4 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:19.469479 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1425/2839] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:44:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:44:19.733649 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fd27c-70c3-4a52-a0da-fffed19fc003 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:19.735187 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-905fd27c-70c3-4a52-a0da-fffed19fc003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:19.748292 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fd27c-70c3-4a52-a0da-fffed19fc003 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:19.753744 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ccaf0-0b24-4499-b875-58df2c974e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:19.755151 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ccaf0-0b24-4499-b875-58df2c974e4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:19.755627 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ccaf0-0b24-4499-b875-58df2c974e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:19.756252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ccaf0-0b24-4499-b875-58df2c974e4c admin admin] PATH_INFO: `/v3/projects/75eb3f6360074168b3382e36594694f4` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:19.757494 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ccaf0-0b24-4499-b875-58df2c974e4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=75eb3f6360074168b3382e36594694f4)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:19.759421 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ccaf0-0b24-4499-b875-58df2c974e4c admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:19.807796 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-ec0ccaf0-0b24-4499-b875-58df2c974e4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75eb3f6360074168b3382e36594694f4', 'request_id': 'req-ec0ccaf0-0b24-4499-b875-58df2c974e4c'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:19.814587 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-ec0ccaf0-0b24-4499-b875-58df2c974e4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75eb3f6360074168b3382e36594694f4', 'request_id': 'req-ec0ccaf0-0b24-4499-b875-58df2c974e4c'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:19.822016 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-ec0ccaf0-0b24-4499-b875-58df2c974e4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75eb3f6360074168b3382e36594694f4', 'request_id': 'req-ec0ccaf0-0b24-4499-b875-58df2c974e4c'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:19.833101 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-ec0ccaf0-0b24-4499-b875-58df2c974e4c admin admin] The token cache is being invalidate because project 75eb3f6360074168b3382e36594694f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:44:19.833306 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-ec0ccaf0-0b24-4499-b875-58df2c974e4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:19.833938 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-ec0ccaf0-0b24-4499-b875-58df2c974e4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:19.835447 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1415/2840] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:44:19 2022] DELETE /identity/v3/projects/75eb3f6360074168b3382e36594694f4 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:44:19.840228 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21000f-754e-48db-8d98-847b6b4f3882 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:19.840518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21000f-754e-48db-8d98-847b6b4f3882 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:19.840824 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21000f-754e-48db-8d98-847b6b4f3882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:19.860465 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-df21000f-754e-48db-8d98-847b6b4f3882 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:19.861707 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-df21000f-754e-48db-8d98-847b6b4f3882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:19.920250 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1426/2841] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:19.927949 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf7068-cc51-4167-a8ea-dfd24684ce1a None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:19.928242 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf7068-cc51-4167-a8ea-dfd24684ce1a None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:19.928524 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf7068-cc51-4167-a8ea-dfd24684ce1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:19.948435 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-10bf7068-cc51-4167-a8ea-dfd24684ce1a None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:19.949698 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-10bf7068-cc51-4167-a8ea-dfd24684ce1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:19.957657 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1416/2842] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:19.961694 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21000f-754e-48db-8d98-847b6b4f3882 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:19.971663 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21000f-754e-48db-8d98-847b6b4f3882 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:19.976233 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39f1a1-dd6d-4c23-8618-64a06a3f8017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:19.976777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39f1a1-dd6d-4c23-8618-64a06a3f8017 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:19.977089 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39f1a1-dd6d-4c23-8618-64a06a3f8017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:19.977364 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39f1a1-dd6d-4c23-8618-64a06a3f8017 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:19.978076 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39f1a1-dd6d-4c23-8618-64a06a3f8017 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:44:19.978381 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39f1a1-dd6d-4c23-8618-64a06a3f8017 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:19.979974 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39f1a1-dd6d-4c23-8618-64a06a3f8017 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:19.991805 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1427/2843] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:44:19.995758 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf7068-cc51-4167-a8ea-dfd24684ce1a None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:20.005601 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf7068-cc51-4167-a8ea-dfd24684ce1a None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:20.010426 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09457213-6fb1-462a-98a0-58f1c83cc1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:20.011038 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09457213-6fb1-462a-98a0-58f1c83cc1a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:20.011325 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09457213-6fb1-462a-98a0-58f1c83cc1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:20.011598 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09457213-6fb1-462a-98a0-58f1c83cc1a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:20.012253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09457213-6fb1-462a-98a0-58f1c83cc1a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:20.014201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09457213-6fb1-462a-98a0-58f1c83cc1a5 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:20.043627 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1417/2844] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:44:19 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:44:20.047745 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39f1a1-dd6d-4c23-8618-64a06a3f8017 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:20.057790 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39f1a1-dd6d-4c23-8618-64a06a3f8017 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:20.097612 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:20.116518 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468328d3-d72e-4db5-9fb3-a6e3a862ca59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:20.117275 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468328d3-d72e-4db5-9fb3-a6e3a862ca59 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:20.117454 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468328d3-d72e-4db5-9fb3-a6e3a862ca59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:20.117690 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468328d3-d72e-4db5-9fb3-a6e3a862ca59 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:20.118348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468328d3-d72e-4db5-9fb3-a6e3a862ca59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:20.120256 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468328d3-d72e-4db5-9fb3-a6e3a862ca59 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:20.141310 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1428/2845] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:44:20 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:44:20.145475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09457213-6fb1-462a-98a0-58f1c83cc1a5 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:20.154403 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:20.158754 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09457213-6fb1-462a-98a0-58f1c83cc1a5 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:20.164789 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911293e1-88c4-472b-b21a-b5a375cbebbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:20.165482 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911293e1-88c4-472b-b21a-b5a375cbebbf admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:20.165786 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911293e1-88c4-472b-b21a-b5a375cbebbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:20.166238 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911293e1-88c4-472b-b21a-b5a375cbebbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:20.167086 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911293e1-88c4-472b-b21a-b5a375cbebbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:20.169527 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911293e1-88c4-472b-b21a-b5a375cbebbf admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:20.179843 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1418/2846] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:44:20.183873 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468328d3-d72e-4db5-9fb3-a6e3a862ca59 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:20.194126 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468328d3-d72e-4db5-9fb3-a6e3a862ca59 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:20.198783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393037c-8327-49bb-8cd1-fa468cf1aa5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:20.199392 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393037c-8327-49bb-8cd1-fa468cf1aa5c admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:20.199676 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393037c-8327-49bb-8cd1-fa468cf1aa5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:20.199953 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393037c-8327-49bb-8cd1-fa468cf1aa5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:20.200650 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393037c-8327-49bb-8cd1-fa468cf1aa5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:20.202286 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393037c-8327-49bb-8cd1-fa468cf1aa5c admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:20.212959 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1429/2847] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:44:20.217442 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911293e1-88c4-472b-b21a-b5a375cbebbf admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:20.227822 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911293e1-88c4-472b-b21a-b5a375cbebbf admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:20.233126 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af544a4-031e-4d29-9d96-ec6582872830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:20.233777 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af544a4-031e-4d29-9d96-ec6582872830 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:20.234060 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af544a4-031e-4d29-9d96-ec6582872830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:20.234337 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af544a4-031e-4d29-9d96-ec6582872830 admin admin] PATH_INFO: `/v3/projects/2779d7fb0f7841b5a57daa762d34451b/users/558a09fd2a614ae2b037912085ae265a/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:20.374398 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af544a4-031e-4d29-9d96-ec6582872830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2779d7fb0f7841b5a57daa762d34451b, user_id=558a09fd2a614ae2b037912085ae265a, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:20.377078 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af544a4-031e-4d29-9d96-ec6582872830 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:20.395069 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1419/2848] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:44:20 2022] PUT /identity/v3/projects/2779d7fb0f7841b5a57daa762d34451b/users/558a09fd2a614ae2b037912085ae265a/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:44:20.413911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf2988-19f7-4ce0-97c1-1b647b04e890 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:20.413911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf2988-19f7-4ce0-97c1-1b647b04e890 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:20.413911 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf2988-19f7-4ce0-97c1-1b647b04e890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:20.451368 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-deaf2988-19f7-4ce0-97c1-1b647b04e890 None None] MFA Rules not processed for user `558a09fd2a614ae2b037912085ae265a`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:20.452570 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-deaf2988-19f7-4ce0-97c1-1b647b04e890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:20.518321 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1430/2849] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:20.529735 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af544a4-031e-4d29-9d96-ec6582872830 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:20.531431 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-1af544a4-031e-4d29-9d96-ec6582872830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:20.591397 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af544a4-031e-4d29-9d96-ec6582872830 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:20.595413 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6395a99-3a3a-44d6-8dd8-dd27c6e09f21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd352d423bff418f932b434c51b15d23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:20.596466 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6395a99-3a3a-44d6-8dd8-dd27c6e09f21 service cinder] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:20.596765 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6395a99-3a3a-44d6-8dd8-dd27c6e09f21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:20.597073 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6395a99-3a3a-44d6-8dd8-dd27c6e09f21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:20.606277 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6395a99-3a3a-44d6-8dd8-dd27c6e09f21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:20.607831 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6395a99-3a3a-44d6-8dd8-dd27c6e09f21 service cinder] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:20.616667 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1420/2850] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:44:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:44:30.450642 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf2988-19f7-4ce0-97c1-1b647b04e890 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:30.512583 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf2988-19f7-4ce0-97c1-1b647b04e890 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:30.517313 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a253c2-1e65-4114-abe0-dc24341e9295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:30.517962 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a253c2-1e65-4114-abe0-dc24341e9295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:30.518247 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a253c2-1e65-4114-abe0-dc24341e9295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:30.518670 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a253c2-1e65-4114-abe0-dc24341e9295 admin admin] PATH_INFO: `/v3/users/558a09fd2a614ae2b037912085ae265a` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:30.519845 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a253c2-1e65-4114-abe0-dc24341e9295 admin admin] RBAC: Authorizing `identity:delete_user(user_id=558a09fd2a614ae2b037912085ae265a)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:30.521806 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a253c2-1e65-4114-abe0-dc24341e9295 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:30.568171 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-06a253c2-1e65-4114-abe0-dc24341e9295 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '558a09fd2a614ae2b037912085ae265a'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:30.572004 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:30.577597 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-06a253c2-1e65-4114-abe0-dc24341e9295 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '558a09fd2a614ae2b037912085ae265a'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:30.591701 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-06a253c2-1e65-4114-abe0-dc24341e9295 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '558a09fd2a614ae2b037912085ae265a'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:30.598850 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-06a253c2-1e65-4114-abe0-dc24341e9295 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '558a09fd2a614ae2b037912085ae265a'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:30.599491 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-06a253c2-1e65-4114-abe0-dc24341e9295 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '558a09fd2a614ae2b037912085ae265a'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:30.605407 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1431/2851] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:44:30 2022] DELETE /identity/v3/users/558a09fd2a614ae2b037912085ae265a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:44:30.609842 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fcce0-dc53-41d7-aef3-a73dc8f29af8 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:30.610302 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fcce0-dc53-41d7-aef3-a73dc8f29af8 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:30.610398 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fcce0-dc53-41d7-aef3-a73dc8f29af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:30.613183 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=39,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:30.631204 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-830fcce0-dc53-41d7-aef3-a73dc8f29af8 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:30.632398 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-830fcce0-dc53-41d7-aef3-a73dc8f29af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:30.689531 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1421/2852] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:30.699766 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a253c2-1e65-4114-abe0-dc24341e9295 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:30.701567 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-06a253c2-1e65-4114-abe0-dc24341e9295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:30.764577 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a253c2-1e65-4114-abe0-dc24341e9295 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:30.768175 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ad2c6-cf17-4473-ba08-18c4ca0d9137 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:30.768786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ad2c6-cf17-4473-ba08-18c4ca0d9137 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:30.769105 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ad2c6-cf17-4473-ba08-18c4ca0d9137 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:30.769375 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ad2c6-cf17-4473-ba08-18c4ca0d9137 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:30.778476 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ad2c6-cf17-4473-ba08-18c4ca0d9137 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:30.779901 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ad2c6-cf17-4473-ba08-18c4ca0d9137 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:30.789425 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1432/2853] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:44:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:44:31.063288 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830fcce0-dc53-41d7-aef3-a73dc8f29af8 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:31.064753 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-830fcce0-dc53-41d7-aef3-a73dc8f29af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:31.076721 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830fcce0-dc53-41d7-aef3-a73dc8f29af8 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:31.082747 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176408a6-a657-4db7-b3c3-750a3089067d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:31.083414 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176408a6-a657-4db7-b3c3-750a3089067d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:31.083698 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176408a6-a657-4db7-b3c3-750a3089067d admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:31.083976 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176408a6-a657-4db7-b3c3-750a3089067d admin admin] PATH_INFO: `/v3/projects/2779d7fb0f7841b5a57daa762d34451b` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:31.085161 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176408a6-a657-4db7-b3c3-750a3089067d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2779d7fb0f7841b5a57daa762d34451b)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:31.087069 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176408a6-a657-4db7-b3c3-750a3089067d admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:31.133214 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-176408a6-a657-4db7-b3c3-750a3089067d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2779d7fb0f7841b5a57daa762d34451b', 'request_id': 'req-176408a6-a657-4db7-b3c3-750a3089067d'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:31.138562 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-176408a6-a657-4db7-b3c3-750a3089067d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2779d7fb0f7841b5a57daa762d34451b', 'request_id': 'req-176408a6-a657-4db7-b3c3-750a3089067d'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:31.145784 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-176408a6-a657-4db7-b3c3-750a3089067d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2779d7fb0f7841b5a57daa762d34451b', 'request_id': 'req-176408a6-a657-4db7-b3c3-750a3089067d'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:31.160189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-176408a6-a657-4db7-b3c3-750a3089067d admin admin] The token cache is being invalidate because project 2779d7fb0f7841b5a57daa762d34451b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:44:31.160538 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-176408a6-a657-4db7-b3c3-750a3089067d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:31.161378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-176408a6-a657-4db7-b3c3-750a3089067d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:31.163431 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1422/2854] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:44:31 2022] DELETE /identity/v3/projects/2779d7fb0f7841b5a57daa762d34451b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:44:40.785120 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=38,keystone:DELETE=2 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:41.157268 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:52.614571 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ad2c6-cf17-4473-ba08-18c4ca0d9137 service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:52.616027 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-777ad2c6-cf17-4473-ba08-18c4ca0d9137 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:52.628752 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ad2c6-cf17-4473-ba08-18c4ca0d9137 service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:52.683650 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9313c-2fc5-4e00-b381-fb6c259708fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:52.684266 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9313c-2fc5-4e00-b381-fb6c259708fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:52.684551 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9313c-2fc5-4e00-b381-fb6c259708fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:52.684824 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9313c-2fc5-4e00-b381-fb6c259708fa admin admin] PATH_INFO: `/v3/users/71f2691b1e75408889d57a01c06a6d16` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:52.686029 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac9313c-2fc5-4e00-b381-fb6c259708fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=71f2691b1e75408889d57a01c06a6d16)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:52.687934 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac9313c-2fc5-4e00-b381-fb6c259708fa admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:52.732941 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-aac9313c-2fc5-4e00-b381-fb6c259708fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71f2691b1e75408889d57a01c06a6d16'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:52.739997 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-aac9313c-2fc5-4e00-b381-fb6c259708fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71f2691b1e75408889d57a01c06a6d16'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:52.751798 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-aac9313c-2fc5-4e00-b381-fb6c259708fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71f2691b1e75408889d57a01c06a6d16'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:52.758522 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-aac9313c-2fc5-4e00-b381-fb6c259708fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71f2691b1e75408889d57a01c06a6d16'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:52.759157 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-aac9313c-2fc5-4e00-b381-fb6c259708fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71f2691b1e75408889d57a01c06a6d16'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:52.765719 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1433/2855] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:44:52 2022] DELETE /identity/v3/users/71f2691b1e75408889d57a01c06a6d16 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:44:52.770207 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73a521-a024-49a9-9f47-ae082fd975b1 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:52.770697 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73a521-a024-49a9-9f47-ae082fd975b1 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:52.770800 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73a521-a024-49a9-9f47-ae082fd975b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:52.790057 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-fb73a521-a024-49a9-9f47-ae082fd975b1 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:52.791259 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-fb73a521-a024-49a9-9f47-ae082fd975b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:52.849012 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1423/2856] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:52.859314 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9313c-2fc5-4e00-b381-fb6c259708fa admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:52.860727 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-aac9313c-2fc5-4e00-b381-fb6c259708fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:52.920651 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9313c-2fc5-4e00-b381-fb6c259708fa admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:52.924240 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45009c44-6400-4eba-abcc-50dd53e9a988 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:52.924850 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45009c44-6400-4eba-abcc-50dd53e9a988 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:52.925160 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45009c44-6400-4eba-abcc-50dd53e9a988 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:52.925433 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45009c44-6400-4eba-abcc-50dd53e9a988 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:52.934330 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45009c44-6400-4eba-abcc-50dd53e9a988 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:52.935744 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45009c44-6400-4eba-abcc-50dd53e9a988 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:52.945319 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1434/2857] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:44:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:44:53.218618 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73a521-a024-49a9-9f47-ae082fd975b1 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:53.220170 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-fb73a521-a024-49a9-9f47-ae082fd975b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:53.233669 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73a521-a024-49a9-9f47-ae082fd975b1 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:53.238303 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582477be-c05d-4171-8936-98ef59ce18f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:53.238946 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582477be-c05d-4171-8936-98ef59ce18f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:53.239685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582477be-c05d-4171-8936-98ef59ce18f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:53.239685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582477be-c05d-4171-8936-98ef59ce18f6 admin admin] PATH_INFO: `/v3/projects/347f7dbd4fd0413e8721edee116bf2a2` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:53.240774 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582477be-c05d-4171-8936-98ef59ce18f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=347f7dbd4fd0413e8721edee116bf2a2)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:53.242715 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582477be-c05d-4171-8936-98ef59ce18f6 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:53.287617 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-582477be-c05d-4171-8936-98ef59ce18f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '347f7dbd4fd0413e8721edee116bf2a2', 'request_id': 'req-582477be-c05d-4171-8936-98ef59ce18f6'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:53.293655 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-582477be-c05d-4171-8936-98ef59ce18f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '347f7dbd4fd0413e8721edee116bf2a2', 'request_id': 'req-582477be-c05d-4171-8936-98ef59ce18f6'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:53.300733 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-582477be-c05d-4171-8936-98ef59ce18f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '347f7dbd4fd0413e8721edee116bf2a2', 'request_id': 'req-582477be-c05d-4171-8936-98ef59ce18f6'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:53.312681 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-582477be-c05d-4171-8936-98ef59ce18f6 admin admin] The token cache is being invalidate because project 347f7dbd4fd0413e8721edee116bf2a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:44:53.313005 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-582477be-c05d-4171-8936-98ef59ce18f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:53.313653 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-582477be-c05d-4171-8936-98ef59ce18f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:44:53.315617 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1424/2858] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:44:53 2022] DELETE /identity/v3/projects/347f7dbd4fd0413e8721edee116bf2a2 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:44:53.320756 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3da960-fa82-4eab-96fb-b656e323c212 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:53.321078 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3da960-fa82-4eab-96fb-b656e323c212 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:53.321357 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3da960-fa82-4eab-96fb-b656e323c212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:53.340574 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-bb3da960-fa82-4eab-96fb-b656e323c212 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:53.341818 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-bb3da960-fa82-4eab-96fb-b656e323c212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:53.359803 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:53.403774 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1435/2859] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:53.411663 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ecece-9c77-4be1-a978-783965580e39 None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:53.411953 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ecece-9c77-4be1-a978-783965580e39 None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:53.412234 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ecece-9c77-4be1-a978-783965580e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:53.431505 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-806ecece-9c77-4be1-a978-783965580e39 None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:53.432703 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-806ecece-9c77-4be1-a978-783965580e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:53.441130 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1425/2860] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:53.445316 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3da960-fa82-4eab-96fb-b656e323c212 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:53.455482 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3da960-fa82-4eab-96fb-b656e323c212 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:53.460071 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344e101-ddc6-4ace-85e2-4282a479f2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:53.460681 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344e101-ddc6-4ace-85e2-4282a479f2d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:53.460995 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344e101-ddc6-4ace-85e2-4282a479f2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:53.461273 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344e101-ddc6-4ace-85e2-4282a479f2d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:53.461960 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a344e101-ddc6-4ace-85e2-4282a479f2d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80109) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 16:44:53.462264 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a344e101-ddc6-4ace-85e2-4282a479f2d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:53.463850 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a344e101-ddc6-4ace-85e2-4282a479f2d7 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:53.475431 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1436/2861] 10.222.0.74 () {58 vars in 1153 bytes} [Wed Jul 27 16:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 16:44:53.479628 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ecece-9c77-4be1-a978-783965580e39 None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:53.488664 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ecece-9c77-4be1-a978-783965580e39 None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:53.493296 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a27dd4-c7b6-48cf-9f84-0e1c6fb49ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:53.493891 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a27dd4-c7b6-48cf-9f84-0e1c6fb49ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:53.494175 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a27dd4-c7b6-48cf-9f84-0e1c6fb49ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:53.494452 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a27dd4-c7b6-48cf-9f84-0e1c6fb49ae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:53.495126 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a27dd4-c7b6-48cf-9f84-0e1c6fb49ae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:53.497567 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a27dd4-c7b6-48cf-9f84-0e1c6fb49ae5 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:53.524533 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1426/2862] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:44:53 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:44:53.528868 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344e101-ddc6-4ace-85e2-4282a479f2d7 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:53.538851 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344e101-ddc6-4ace-85e2-4282a479f2d7 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:53.593421 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61398fed-555b-4222-a8a8-b9a866ab4844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:53.594028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61398fed-555b-4222-a8a8-b9a866ab4844 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:53.594307 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61398fed-555b-4222-a8a8-b9a866ab4844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:53.594579 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61398fed-555b-4222-a8a8-b9a866ab4844 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:53.595239 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61398fed-555b-4222-a8a8-b9a866ab4844 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:53.597426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61398fed-555b-4222-a8a8-b9a866ab4844 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:53.618063 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1437/2863] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:44:53 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 16:44:53.622974 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a27dd4-c7b6-48cf-9f84-0e1c6fb49ae5 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:53.632979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a27dd4-c7b6-48cf-9f84-0e1c6fb49ae5 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:53.637486 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625b520-ba28-4e3f-92c2-950f1ce9fcec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:53.638084 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625b520-ba28-4e3f-92c2-950f1ce9fcec admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:53.638459 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625b520-ba28-4e3f-92c2-950f1ce9fcec admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:53.638633 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625b520-ba28-4e3f-92c2-950f1ce9fcec admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:53.639464 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625b520-ba28-4e3f-92c2-950f1ce9fcec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:53.641402 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625b520-ba28-4e3f-92c2-950f1ce9fcec admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:53.651228 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1427/2864] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:44:53.655343 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61398fed-555b-4222-a8a8-b9a866ab4844 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:53.665092 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61398fed-555b-4222-a8a8-b9a866ab4844 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:53.670230 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bfd2b8-c3f3-44ec-9a0a-3e1d52c5ce91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:53.670814 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfd2b8-c3f3-44ec-9a0a-3e1d52c5ce91 admin admin] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:53.671095 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfd2b8-c3f3-44ec-9a0a-3e1d52c5ce91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:53.671368 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfd2b8-c3f3-44ec-9a0a-3e1d52c5ce91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:53.672047 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bfd2b8-c3f3-44ec-9a0a-3e1d52c5ce91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:53.673677 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bfd2b8-c3f3-44ec-9a0a-3e1d52c5ce91 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:53.684516 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1438/2865] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:44:53.688475 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625b520-ba28-4e3f-92c2-950f1ce9fcec admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:53.698000 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625b520-ba28-4e3f-92c2-950f1ce9fcec admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:53.702611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a1006-6d48-4398-901f-8b560f06eaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:53.703243 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a1006-6d48-4398-901f-8b560f06eaa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:53.703520 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a1006-6d48-4398-901f-8b560f06eaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:53.703790 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a1006-6d48-4398-901f-8b560f06eaa8 admin admin] PATH_INFO: `/v3/projects/51a4d6fb95404988996ef7680ac1829e/users/a5cc5606ede347b3894855fed8f49fb8/roles/8de48592893c43789e1d40158943fd52` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:53.717720 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a1006-6d48-4398-901f-8b560f06eaa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51a4d6fb95404988996ef7680ac1829e, user_id=a5cc5606ede347b3894855fed8f49fb8, role_id=8de48592893c43789e1d40158943fd52)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:53.720335 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a1006-6d48-4398-901f-8b560f06eaa8 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:53.733209 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1428/2866] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:44:53 2022] PUT /identity/v3/projects/51a4d6fb95404988996ef7680ac1829e/users/a5cc5606ede347b3894855fed8f49fb8/roles/8de48592893c43789e1d40158943fd52 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:44:53.738011 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7ec3c-6c14-403b-84bd-fb833e22ae7b None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:53.738297 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7ec3c-6c14-403b-84bd-fb833e22ae7b None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:53.738573 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7ec3c-6c14-403b-84bd-fb833e22ae7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:53.759253 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-bfc7ec3c-6c14-403b-84bd-fb833e22ae7b None None] MFA Rules not processed for user `2682f5470265422cbaaa91dfd16e8ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:53.760442 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-bfc7ec3c-6c14-403b-84bd-fb833e22ae7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:53.819151 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1439/2867] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:53.828653 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a1006-6d48-4398-901f-8b560f06eaa8 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:53.830373 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-075a1006-6d48-4398-901f-8b560f06eaa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:53.851939 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:53.888847 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a1006-6d48-4398-901f-8b560f06eaa8 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:53.892493 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1296aa-ee32-46d1-805c-5d6ab1cbbb6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:53.893130 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1296aa-ee32-46d1-805c-5d6ab1cbbb6b service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:53.893406 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1296aa-ee32-46d1-805c-5d6ab1cbbb6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:53.893673 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1296aa-ee32-46d1-805c-5d6ab1cbbb6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:53.903017 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1296aa-ee32-46d1-805c-5d6ab1cbbb6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:53.904435 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1296aa-ee32-46d1-805c-5d6ab1cbbb6b service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:53.914029 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1429/2868] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:44:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:44:57.902772 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fef66a-4a7f-4c40-8bab-6a3a06ff7019 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:57.903334 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fef66a-4a7f-4c40-8bab-6a3a06ff7019 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:57.903334 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fef66a-4a7f-4c40-8bab-6a3a06ff7019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:57.937552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-06fef66a-4a7f-4c40-8bab-6a3a06ff7019 None None] MFA Rules not processed for user `a5cc5606ede347b3894855fed8f49fb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:44:57.938943 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-06fef66a-4a7f-4c40-8bab-6a3a06ff7019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:44:57.957542 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:44:58.009835 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1440/2869] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:44:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:44:58.014929 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1296aa-ee32-46d1-805c-5d6ab1cbbb6b service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:58.025504 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1296aa-ee32-46d1-805c-5d6ab1cbbb6b service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:58.031258 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ca54f-b093-4f07-9099-ec572c365e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:58.032284 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ca54f-b093-4f07-9099-ec572c365e01 admin admin] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:58.032444 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ca54f-b093-4f07-9099-ec572c365e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:58.032736 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ca54f-b093-4f07-9099-ec572c365e01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:58.037333 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ca54f-b093-4f07-9099-ec572c365e01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:58.039261 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ca54f-b093-4f07-9099-ec572c365e01 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:58.067537 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1430/2870] 10.222.0.74 () {60 vars in 1153 bytes} [Wed Jul 27 16:44:58 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:44:58.071617 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fef66a-4a7f-4c40-8bab-6a3a06ff7019 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:58.083416 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fef66a-4a7f-4c40-8bab-6a3a06ff7019 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:58.143481 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223e0eb-05f5-457b-a328-ca5591aeca16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:58.144208 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223e0eb-05f5-457b-a328-ca5591aeca16 admin admin] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:58.144485 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223e0eb-05f5-457b-a328-ca5591aeca16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:58.144757 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223e0eb-05f5-457b-a328-ca5591aeca16 admin admin] PATH_INFO: `/v3/users` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:58.145483 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d223e0eb-05f5-457b-a328-ca5591aeca16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:58.147406 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d223e0eb-05f5-457b-a328-ca5591aeca16 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:58.170620 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1441/2871] 10.222.0.74 () {60 vars in 1144 bytes} [Wed Jul 27 16:44:58 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 16:44:58.174501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ca54f-b093-4f07-9099-ec572c365e01 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:58.185623 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ca54f-b093-4f07-9099-ec572c365e01 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:58.190252 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdc8ce-cb16-4092-82d8-bec72c27721c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:58.190874 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdc8ce-cb16-4092-82d8-bec72c27721c admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:58.191156 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdc8ce-cb16-4092-82d8-bec72c27721c admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:58.191427 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdc8ce-cb16-4092-82d8-bec72c27721c admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:58.192139 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bdc8ce-cb16-4092-82d8-bec72c27721c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:58.193794 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bdc8ce-cb16-4092-82d8-bec72c27721c admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:58.210383 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1431/2872] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 16:44:58.214836 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223e0eb-05f5-457b-a328-ca5591aeca16 admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:58.226302 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223e0eb-05f5-457b-a328-ca5591aeca16 admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:58.233848 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df98998-e047-4170-9e27-e114822948fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:58.234786 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df98998-e047-4170-9e27-e114822948fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:58.235229 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df98998-e047-4170-9e27-e114822948fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:58.235664 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df98998-e047-4170-9e27-e114822948fc admin admin] PATH_INFO: `/v3/projects/6ded7235141846d9b31133f328118279/users/08f61e8c72c4440d90fb8b46c4f94c64/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:58.253727 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df98998-e047-4170-9e27-e114822948fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ded7235141846d9b31133f328118279, user_id=08f61e8c72c4440d90fb8b46c4f94c64, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:58.256463 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df98998-e047-4170-9e27-e114822948fc admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:58.273118 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1442/2873] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:44:58 2022] PUT /identity/v3/projects/6ded7235141846d9b31133f328118279/users/08f61e8c72c4440d90fb8b46c4f94c64/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:44:58.277279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdc8ce-cb16-4092-82d8-bec72c27721c admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:58.287905 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdc8ce-cb16-4092-82d8-bec72c27721c admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:58.346371 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc8b661-4a1c-43d9-8626-34ed4f44fc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:58.347240 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc8b661-4a1c-43d9-8626-34ed4f44fc7e admin admin] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:58.347518 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc8b661-4a1c-43d9-8626-34ed4f44fc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:58.347786 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc8b661-4a1c-43d9-8626-34ed4f44fc7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:58.348463 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc8b661-4a1c-43d9-8626-34ed4f44fc7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:58.350087 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc8b661-4a1c-43d9-8626-34ed4f44fc7e admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:58.359904 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1432/2874] 10.222.0.74 () {58 vars in 1122 bytes} [Wed Jul 27 16:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 16:44:58.364216 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df98998-e047-4170-9e27-e114822948fc admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:44:58.375065 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df98998-e047-4170-9e27-e114822948fc admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:44:58.379721 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ccee3-e71c-4eb7-95bd-05e86adb92ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:44:58.380380 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ccee3-e71c-4eb7-95bd-05e86adb92ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:44:58.380665 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ccee3-e71c-4eb7-95bd-05e86adb92ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:44:58.380979 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ccee3-e71c-4eb7-95bd-05e86adb92ed admin admin] PATH_INFO: `/v3/projects/6ded7235141846d9b31133f328118279/users/08f61e8c72c4440d90fb8b46c4f94c64/roles/ca37e43082fa40c6a88b12680a0ca94b` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:44:58.382940 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ccee3-e71c-4eb7-95bd-05e86adb92ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ded7235141846d9b31133f328118279, user_id=08f61e8c72c4440d90fb8b46c4f94c64, role_id=ca37e43082fa40c6a88b12680a0ca94b)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:44:58.385637 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ccee3-e71c-4eb7-95bd-05e86adb92ed admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:44:58.396423 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1443/2875] 10.222.0.74 () {60 vars in 1483 bytes} [Wed Jul 27 16:44:58 2022] PUT /identity/v3/projects/6ded7235141846d9b31133f328118279/users/08f61e8c72c4440d90fb8b46c4f94c64/roles/ca37e43082fa40c6a88b12680a0ca94b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:45:03.008189 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d4aa1-f930-4823-b070-ec49a95c7c2e None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:45:03.009158 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d4aa1-f930-4823-b070-ec49a95c7c2e None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:45:03.009597 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d4aa1-f930-4823-b070-ec49a95c7c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:45:03.068694 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-263d4aa1-f930-4823-b070-ec49a95c7c2e None None] MFA Rules not processed for user `08f61e8c72c4440d90fb8b46c4f94c64`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:45:03.070201 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-263d4aa1-f930-4823-b070-ec49a95c7c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:45:03.185068 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1433/2876] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:45:03 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:45:03.201772 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ccee3-e71c-4eb7-95bd-05e86adb92ed admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:45:03.203549 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-7a7ccee3-e71c-4eb7-95bd-05e86adb92ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:45:03.303926 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ccee3-e71c-4eb7-95bd-05e86adb92ed admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:45:03.309797 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d790bf-e075-4f68-878d-1643aa952eb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:45:03.310552 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d790bf-e075-4f68-878d-1643aa952eb5 service nova] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:45:03.310935 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d790bf-e075-4f68-878d-1643aa952eb5 service nova] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:45:03.311313 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d790bf-e075-4f68-878d-1643aa952eb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:45:03.317137 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:45:03.376325 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d790bf-e075-4f68-878d-1643aa952eb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:45:03.378210 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d790bf-e075-4f68-878d-1643aa952eb5 service nova] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:45:03.388589 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1444/2877] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:45:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:45:04.351636 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbb641-b653-432b-b5eb-47bdf89808ac None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:45:04.352180 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbb641-b653-432b-b5eb-47bdf89808ac None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:45:04.352180 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbb641-b653-432b-b5eb-47bdf89808ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:45:04.371769 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-64fbb641-b653-432b-b5eb-47bdf89808ac None None] MFA Rules not processed for user `a5cc5606ede347b3894855fed8f49fb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:45:04.372987 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-64fbb641-b653-432b-b5eb-47bdf89808ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:45:04.379795 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1434/2878] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:45:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:45:13.383820 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=24 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:45:14.366161 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:45:32.317498 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d790bf-e075-4f68-878d-1643aa952eb5 service nova] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:45:32.319414 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-f0d790bf-e075-4f68-878d-1643aa952eb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:45:32.376783 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d790bf-e075-4f68-878d-1643aa952eb5 service nova] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:45:32.380489 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88712a73-e595-4cf2-a2b8-3e3f1cedfac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:45:32.381143 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88712a73-e595-4cf2-a2b8-3e3f1cedfac7 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:45:32.381426 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88712a73-e595-4cf2-a2b8-3e3f1cedfac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:45:32.381705 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88712a73-e595-4cf2-a2b8-3e3f1cedfac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:45:32.390761 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88712a73-e595-4cf2-a2b8-3e3f1cedfac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:45:32.392192 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88712a73-e595-4cf2-a2b8-3e3f1cedfac7 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:45:32.401484 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1445/2879] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:45:32 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:45:42.397365 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=26 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:46:07.655693 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbb641-b653-432b-b5eb-47bdf89808ac None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:46:07.665448 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbb641-b653-432b-b5eb-47bdf89808ac None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:46:07.669001 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f988f6c-1fb1-4eb5-b5ce-abd26e3e5007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:46:07.669598 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f988f6c-1fb1-4eb5-b5ce-abd26e3e5007 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:46:07.669892 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f988f6c-1fb1-4eb5-b5ce-abd26e3e5007 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:46:07.670157 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f988f6c-1fb1-4eb5-b5ce-abd26e3e5007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:46:07.671855 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-4f988f6c-1fb1-4eb5-b5ce-abd26e3e5007 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:46:07.685343 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f988f6c-1fb1-4eb5-b5ce-abd26e3e5007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:46:07.686742 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f988f6c-1fb1-4eb5-b5ce-abd26e3e5007 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:46:07.698021 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1435/2880] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:46:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:46:08.879068 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88712a73-e595-4cf2-a2b8-3e3f1cedfac7 service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:46:08.887929 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88712a73-e595-4cf2-a2b8-3e3f1cedfac7 service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:46:08.891562 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395a9c4-eeae-42ab-83a0-af3dcef736f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:46:08.892164 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395a9c4-eeae-42ab-83a0-af3dcef736f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:46:08.892498 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395a9c4-eeae-42ab-83a0-af3dcef736f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:46:08.892726 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395a9c4-eeae-42ab-83a0-af3dcef736f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:46:08.894469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-2395a9c4-eeae-42ab-83a0-af3dcef736f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:46:08.907762 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395a9c4-eeae-42ab-83a0-af3dcef736f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:46:08.909211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395a9c4-eeae-42ab-83a0-af3dcef736f3 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:46:08.920130 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1446/2881] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:46:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:46:11.193993 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d5cb2-9bad-4335-a02a-4e83ff09154e None None] REQUEST_METHOD: `POST` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:46:11.194465 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d5cb2-9bad-4335-a02a-4e83ff09154e None None] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:46:11.194593 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d5cb2-9bad-4335-a02a-4e83ff09154e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:46:11.213620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.auth.core [None req-9a7d5cb2-9bad-4335-a02a-4e83ff09154e None None] MFA Rules not processed for user `f965c9163ebf41a9866af7389feaf17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:46:11.214824 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-9a7d5cb2-9bad-4335-a02a-4e83ff09154e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:46:11.284512 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1436/2882] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:46:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 16:46:18.915348 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=6 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:46:21.277563 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=31 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:46:27.873404 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395a9c4-eeae-42ab-83a0-af3dcef736f3 service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:46:27.874897 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-2395a9c4-eeae-42ab-83a0-af3dcef736f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:46:27.888998 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395a9c4-eeae-42ab-83a0-af3dcef736f3 service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:46:27.895159 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e75fdd-b560-4e51-b376-49a64053235a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:46:27.895775 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e75fdd-b560-4e51-b376-49a64053235a service nova] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:46:27.896092 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e75fdd-b560-4e51-b376-49a64053235a service nova] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:46:27.896368 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e75fdd-b560-4e51-b376-49a64053235a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:46:27.898142 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-51e75fdd-b560-4e51-b376-49a64053235a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:46:27.911162 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e75fdd-b560-4e51-b376-49a64053235a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:46:27.912781 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e75fdd-b560-4e51-b376-49a64053235a service nova] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:46:27.923666 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1447/2883] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:46:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:46:27.988378 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d5cb2-9bad-4335-a02a-4e83ff09154e None None] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:46:27.998506 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d5cb2-9bad-4335-a02a-4e83ff09154e None None] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:46:28.003732 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65660c-7a1f-4add-b55b-40aaf46059e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:46:28.004595 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65660c-7a1f-4add-b55b-40aaf46059e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:46:28.004880 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65660c-7a1f-4add-b55b-40aaf46059e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:46:28.005184 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65660c-7a1f-4add-b55b-40aaf46059e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:46:28.006968 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-dc65660c-7a1f-4add-b55b-40aaf46059e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:46:28.017405 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65660c-7a1f-4add-b55b-40aaf46059e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:46:28.018828 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65660c-7a1f-4add-b55b-40aaf46059e6 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:46:28.026886 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1437/2884] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:46:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:46:37.919100 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=6 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:46:38.023204 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=6 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:47:43.161164 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e75fdd-b560-4e51-b376-49a64053235a service nova] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:47:43.170930 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e75fdd-b560-4e51-b376-49a64053235a service nova] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:47:43.174543 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2c659-725f-436a-b6b5-f09da7ec0fcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:47:43.175153 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2c659-725f-436a-b6b5-f09da7ec0fcd service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:47:43.175435 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2c659-725f-436a-b6b5-f09da7ec0fcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:47:43.175710 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2c659-725f-436a-b6b5-f09da7ec0fcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:47:43.177466 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-c8e2c659-725f-436a-b6b5-f09da7ec0fcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:47:43.188125 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2c659-725f-436a-b6b5-f09da7ec0fcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:47:43.189593 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2c659-725f-436a-b6b5-f09da7ec0fcd service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:47:43.197933 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1448/2885] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:47:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:47:43.277911 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65660c-7a1f-4add-b55b-40aaf46059e6 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:47:43.286560 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65660c-7a1f-4add-b55b-40aaf46059e6 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:47:43.290131 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2077602-56dc-4882-a9aa-54308504b9d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:47:43.290733 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2077602-56dc-4882-a9aa-54308504b9d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:47:43.291015 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2077602-56dc-4882-a9aa-54308504b9d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:47:43.291286 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2077602-56dc-4882-a9aa-54308504b9d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:47:43.293582 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-f2077602-56dc-4882-a9aa-54308504b9d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:47:43.307574 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2077602-56dc-4882-a9aa-54308504b9d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:47:43.309645 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2077602-56dc-4882-a9aa-54308504b9d8 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:47:43.320517 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1438/2886] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:47:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:47:43.383537 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2c659-725f-436a-b6b5-f09da7ec0fcd service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:47:43.384989 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-c8e2c659-725f-436a-b6b5-f09da7ec0fcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:47:43.454760 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2c659-725f-436a-b6b5-f09da7ec0fcd service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:47:43.459730 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9d532-3eb1-4193-a78e-3ec18172ec8b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f8e0f82a8541aead6cd8cc45f4a86b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:47:43.460345 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9d532-3eb1-4193-a78e-3ec18172ec8b service placement] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:47:43.460630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9d532-3eb1-4193-a78e-3ec18172ec8b service placement] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:47:43.460933 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9d532-3eb1-4193-a78e-3ec18172ec8b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:47:43.462670 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-fdf9d532-3eb1-4193-a78e-3ec18172ec8b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:47:43.474221 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9d532-3eb1-4193-a78e-3ec18172ec8b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:47:43.475801 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9d532-3eb1-4193-a78e-3ec18172ec8b service placement] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:47:43.486359 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1449/2887] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:47:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:47:46.450915 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2077602-56dc-4882-a9aa-54308504b9d8 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:47:46.462698 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2077602-56dc-4882-a9aa-54308504b9d8 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:47:46.468180 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffbe6a-161a-4e96-8813-7fd65593d5b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:47:46.468817 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffbe6a-161a-4e96-8813-7fd65593d5b4 service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:47:46.469155 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffbe6a-161a-4e96-8813-7fd65593d5b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:47:46.469429 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffbe6a-161a-4e96-8813-7fd65593d5b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:47:46.471418 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-19ffbe6a-161a-4e96-8813-7fd65593d5b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:47:46.484015 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffbe6a-161a-4e96-8813-7fd65593d5b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:47:46.485611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffbe6a-161a-4e96-8813-7fd65593d5b4 service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:47:46.496969 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1439/2888] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:47:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:47:53.481983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=38 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:47:56.491819 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=12 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:48:01.770804 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256929db-a63e-4d71-9ff1-1b4669416436 None None] REQUEST_METHOD: `POST` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:48:01.771330 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256929db-a63e-4d71-9ff1-1b4669416436 None None] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:48:01.771454 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256929db-a63e-4d71-9ff1-1b4669416436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:48:01.791304 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.auth.core [None req-256929db-a63e-4d71-9ff1-1b4669416436 None None] MFA Rules not processed for user `f965c9163ebf41a9866af7389feaf17c`. Rule list: `[]` (Enabled: `True`). {{(pid=80109) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 16:48:01.793321 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-256929db-a63e-4d71-9ff1-1b4669416436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:48:01.800438 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1450/2889] 10.222.0.74 () {58 vars in 958 bytes} [Wed Jul 27 16:48:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 16:48:11.785983 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=3 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:48:19.538813 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffbe6a-161a-4e96-8813-7fd65593d5b4 service nova] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:48:19.549949 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffbe6a-161a-4e96-8813-7fd65593d5b4 service nova] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:48:19.555814 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5c888-9216-426c-a5e9-236492048633 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56dfafd15e284b7986ba07dd3819bf65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:48:19.556437 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5c888-9216-426c-a5e9-236492048633 service nova] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:48:19.556723 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5c888-9216-426c-a5e9-236492048633 service nova] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:48:19.557062 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5c888-9216-426c-a5e9-236492048633 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:48:19.558832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ded5c888-9216-426c-a5e9-236492048633 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:48:19.571372 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5c888-9216-426c-a5e9-236492048633 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:48:19.572973 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5c888-9216-426c-a5e9-236492048633 service nova] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:48:19.584987 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1440/2890] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:48:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:48:29.580399 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=6 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:49:09.196044 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256929db-a63e-4d71-9ff1-1b4669416436 None None] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:49:09.209041 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256929db-a63e-4d71-9ff1-1b4669416436 None None] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:49:09.214203 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9dfa1-c531-4af1-97ff-1351480bf7ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f8e0f82a8541aead6cd8cc45f4a86b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:49:09.214816 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9dfa1-c531-4af1-97ff-1351480bf7ea service placement] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:49:09.215098 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9dfa1-c531-4af1-97ff-1351480bf7ea service placement] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:49:09.215373 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9dfa1-c531-4af1-97ff-1351480bf7ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:49:09.217250 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-61d9dfa1-c531-4af1-97ff-1351480bf7ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:49:09.229028 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9dfa1-c531-4af1-97ff-1351480bf7ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:49:09.230628 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9dfa1-c531-4af1-97ff-1351480bf7ea service placement] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:49:09.240511 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1451/2891] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:49:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:49:14.031620 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5c888-9216-426c-a5e9-236492048633 service nova] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:49:14.036772 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-ded5c888-9216-426c-a5e9-236492048633 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:49:14.054513 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5c888-9216-426c-a5e9-236492048633 service nova] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:49:14.062646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f264c-02f1-482a-9138-659264e83a53 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f8e0f82a8541aead6cd8cc45f4a86b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:49:14.063279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f264c-02f1-482a-9138-659264e83a53 service placement] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:49:14.063746 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f264c-02f1-482a-9138-659264e83a53 service placement] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:49:14.064213 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f264c-02f1-482a-9138-659264e83a53 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:49:14.066050 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-3d6f264c-02f1-482a-9138-659264e83a53 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:49:14.080240 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f264c-02f1-482a-9138-659264e83a53 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:49:14.082018 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f264c-02f1-482a-9138-659264e83a53 service placement] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:49:14.097281 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1441/2892] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:49:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:49:17.195693 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9dfa1-c531-4af1-97ff-1351480bf7ea service placement] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:49:17.206702 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9dfa1-c531-4af1-97ff-1351480bf7ea service placement] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:49:17.211828 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656f56a-d3a8-4493-a36f-a6da8f5acb9e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f8e0f82a8541aead6cd8cc45f4a86b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:49:17.212530 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656f56a-d3a8-4493-a36f-a6da8f5acb9e service placement] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:49:17.212882 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656f56a-d3a8-4493-a36f-a6da8f5acb9e service placement] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:49:17.213444 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656f56a-d3a8-4493-a36f-a6da8f5acb9e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:49:17.215301 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-d656f56a-d3a8-4493-a36f-a6da8f5acb9e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:49:17.227366 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656f56a-d3a8-4493-a36f-a6da8f5acb9e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:49:17.228987 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656f56a-d3a8-4493-a36f-a6da8f5acb9e service placement] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:49:17.240213 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1452/2893] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:49:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:49:17.261692 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f264c-02f1-482a-9138-659264e83a53 service placement] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:49:17.335267 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f264c-02f1-482a-9138-659264e83a53 service placement] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:49:17.340636 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9994d9d-a252-4b78-acdc-3105bad69ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:49:17.341320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9994d9d-a252-4b78-acdc-3105bad69ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:49:17.341572 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9994d9d-a252-4b78-acdc-3105bad69ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:49:17.341840 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9994d9d-a252-4b78-acdc-3105bad69ea9 admin admin] PATH_INFO: `/v3/users/a5cc5606ede347b3894855fed8f49fb8` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:49:17.342979 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9994d9d-a252-4b78-acdc-3105bad69ea9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5cc5606ede347b3894855fed8f49fb8)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:49:17.345425 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9994d9d-a252-4b78-acdc-3105bad69ea9 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:49:17.391994 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-d9994d9d-a252-4b78-acdc-3105bad69ea9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5cc5606ede347b3894855fed8f49fb8'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:49:17.399349 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-d9994d9d-a252-4b78-acdc-3105bad69ea9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5cc5606ede347b3894855fed8f49fb8'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:49:17.407245 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-d9994d9d-a252-4b78-acdc-3105bad69ea9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5cc5606ede347b3894855fed8f49fb8'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:49:17.422184 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-d9994d9d-a252-4b78-acdc-3105bad69ea9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5cc5606ede347b3894855fed8f49fb8'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:49:17.422898 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-d9994d9d-a252-4b78-acdc-3105bad69ea9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5cc5606ede347b3894855fed8f49fb8'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:49:17.428799 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1442/2894] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:49:17 2022] DELETE /identity/v3/users/a5cc5606ede347b3894855fed8f49fb8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:49:17.613790 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656f56a-d3a8-4493-a36f-a6da8f5acb9e service placement] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:49:17.624842 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656f56a-d3a8-4493-a36f-a6da8f5acb9e service placement] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:49:17.682884 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d47b8-374f-4779-a472-be33cc94c0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:49:17.683569 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d47b8-374f-4779-a472-be33cc94c0fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:49:17.683847 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d47b8-374f-4779-a472-be33cc94c0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:49:17.684121 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d47b8-374f-4779-a472-be33cc94c0fa admin admin] PATH_INFO: `/v3/projects/51a4d6fb95404988996ef7680ac1829e` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:49:17.685395 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364d47b8-374f-4779-a472-be33cc94c0fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=51a4d6fb95404988996ef7680ac1829e)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:49:17.687485 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364d47b8-374f-4779-a472-be33cc94c0fa admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:49:17.733952 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-364d47b8-374f-4779-a472-be33cc94c0fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51a4d6fb95404988996ef7680ac1829e', 'request_id': 'req-364d47b8-374f-4779-a472-be33cc94c0fa'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:49:17.741211 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-364d47b8-374f-4779-a472-be33cc94c0fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51a4d6fb95404988996ef7680ac1829e', 'request_id': 'req-364d47b8-374f-4779-a472-be33cc94c0fa'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:49:17.747973 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-364d47b8-374f-4779-a472-be33cc94c0fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51a4d6fb95404988996ef7680ac1829e', 'request_id': 'req-364d47b8-374f-4779-a472-be33cc94c0fa'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:49:17.760331 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-364d47b8-374f-4779-a472-be33cc94c0fa admin admin] The token cache is being invalidate because project 51a4d6fb95404988996ef7680ac1829e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:49:17.760620 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-364d47b8-374f-4779-a472-be33cc94c0fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:49:17.762095 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-364d47b8-374f-4779-a472-be33cc94c0fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:49:17.764381 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1453/2895] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:49:17 2022] DELETE /identity/v3/projects/51a4d6fb95404988996ef7680ac1829e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:49:17.768965 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9994d9d-a252-4b78-acdc-3105bad69ea9 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:49:17.770408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d9994d9d-a252-4b78-acdc-3105bad69ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:49:17.783953 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9994d9d-a252-4b78-acdc-3105bad69ea9 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:49:17.842408 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8726f1-e468-4e03-b6df-f0f35d091883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:49:17.843056 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8726f1-e468-4e03-b6df-f0f35d091883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:49:17.843594 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8726f1-e468-4e03-b6df-f0f35d091883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:49:17.843873 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8726f1-e468-4e03-b6df-f0f35d091883 admin admin] PATH_INFO: `/v3/users/08f61e8c72c4440d90fb8b46c4f94c64` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:49:17.845092 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8726f1-e468-4e03-b6df-f0f35d091883 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08f61e8c72c4440d90fb8b46c4f94c64)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:49:17.847011 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8726f1-e468-4e03-b6df-f0f35d091883 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:49:17.852345 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:49:17.895359 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-aa8726f1-e468-4e03-b6df-f0f35d091883 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08f61e8c72c4440d90fb8b46c4f94c64'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:49:17.902827 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-aa8726f1-e468-4e03-b6df-f0f35d091883 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08f61e8c72c4440d90fb8b46c4f94c64'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:49:17.910388 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-aa8726f1-e468-4e03-b6df-f0f35d091883 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08f61e8c72c4440d90fb8b46c4f94c64'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:49:17.922423 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-aa8726f1-e468-4e03-b6df-f0f35d091883 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08f61e8c72c4440d90fb8b46c4f94c64'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:49:17.923140 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-aa8726f1-e468-4e03-b6df-f0f35d091883 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08f61e8c72c4440d90fb8b46c4f94c64'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:49:17.929287 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1443/2896] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:49:17 2022] DELETE /identity/v3/users/08f61e8c72c4440d90fb8b46c4f94c64 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:49:18.116669 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d47b8-374f-4779-a472-be33cc94c0fa admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:49:18.118190 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-364d47b8-374f-4779-a472-be33cc94c0fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:49:18.131445 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d47b8-374f-4779-a472-be33cc94c0fa admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:49:18.189133 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf7d75-bdfe-4012-8972-226bdc23ff1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:49:18.189777 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf7d75-bdfe-4012-8972-226bdc23ff1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:49:18.190058 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf7d75-bdfe-4012-8972-226bdc23ff1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:49:18.190332 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf7d75-bdfe-4012-8972-226bdc23ff1e admin admin] PATH_INFO: `/v3/projects/6ded7235141846d9b31133f328118279` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:49:18.191472 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf7d75-bdfe-4012-8972-226bdc23ff1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ded7235141846d9b31133f328118279)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:49:18.193405 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf7d75-bdfe-4012-8972-226bdc23ff1e admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:49:18.200032 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:49:18.241030 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-27bf7d75-bdfe-4012-8972-226bdc23ff1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ded7235141846d9b31133f328118279', 'request_id': 'req-27bf7d75-bdfe-4012-8972-226bdc23ff1e'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:49:18.247844 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-27bf7d75-bdfe-4012-8972-226bdc23ff1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ded7235141846d9b31133f328118279', 'request_id': 'req-27bf7d75-bdfe-4012-8972-226bdc23ff1e'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:49:18.253390 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-27bf7d75-bdfe-4012-8972-226bdc23ff1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ded7235141846d9b31133f328118279', 'request_id': 'req-27bf7d75-bdfe-4012-8972-226bdc23ff1e'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:49:18.264734 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-27bf7d75-bdfe-4012-8972-226bdc23ff1e admin admin] The token cache is being invalidate because project 6ded7235141846d9b31133f328118279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:49:18.265062 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-27bf7d75-bdfe-4012-8972-226bdc23ff1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:49:18.265722 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-27bf7d75-bdfe-4012-8972-226bdc23ff1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:49:18.267236 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1454/2897] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:49:18 2022] DELETE /identity/v3/projects/6ded7235141846d9b31133f328118279 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:49:27.925685 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:49:28.261148 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:49:58.364414 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8726f1-e468-4e03-b6df-f0f35d091883 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:49:58.365850 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-aa8726f1-e468-4e03-b6df-f0f35d091883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:49:58.422764 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8726f1-e468-4e03-b6df-f0f35d091883 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:49:58.469546 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496420e-5ed3-448b-beda-d6c81d1ccc49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:49:58.470157 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496420e-5ed3-448b-beda-d6c81d1ccc49 service neutron] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:49:58.470439 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496420e-5ed3-448b-beda-d6c81d1ccc49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:49:58.470710 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496420e-5ed3-448b-beda-d6c81d1ccc49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:49:58.472422 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d496420e-5ed3-448b-beda-d6c81d1ccc49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:49:58.527995 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496420e-5ed3-448b-beda-d6c81d1ccc49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:49:58.529514 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496420e-5ed3-448b-beda-d6c81d1ccc49 service neutron] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:49:58.538320 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1444/2898] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:49:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:50:08.534216 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=68 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:50:11.848846 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf7d75-bdfe-4012-8972-226bdc23ff1e admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:50:11.850593 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-27bf7d75-bdfe-4012-8972-226bdc23ff1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:50:11.863852 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf7d75-bdfe-4012-8972-226bdc23ff1e admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:50:11.867376 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9339fa-ee84-44a9-9bf5-c210dc9458d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '193dd9fee1844275ad0dab602e93df11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:50:11.868063 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9339fa-ee84-44a9-9bf5-c210dc9458d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:50:11.868283 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9339fa-ee84-44a9-9bf5-c210dc9458d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:50:11.868553 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9339fa-ee84-44a9-9bf5-c210dc9458d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:50:11.870264 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-9a9339fa-ee84-44a9-9bf5-c210dc9458d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:50:11.931469 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9339fa-ee84-44a9-9bf5-c210dc9458d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:50:11.932896 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9339fa-ee84-44a9-9bf5-c210dc9458d4 service neutron] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:50:11.942342 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1455/2899] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:50:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 16:50:17.960611 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496420e-5ed3-448b-beda-d6c81d1ccc49 service neutron] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:50:17.962607 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d496420e-5ed3-448b-beda-d6c81d1ccc49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:50:17.976322 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496420e-5ed3-448b-beda-d6c81d1ccc49 service neutron] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:50:17.981581 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c5e73-9251-4b1d-b443-63e57e2aac55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:50:17.982217 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c5e73-9251-4b1d-b443-63e57e2aac55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:50:17.982501 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c5e73-9251-4b1d-b443-63e57e2aac55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:50:17.982779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c5e73-9251-4b1d-b443-63e57e2aac55 admin admin] PATH_INFO: `/v3/users/f965c9163ebf41a9866af7389feaf17c` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:50:17.983944 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c5e73-9251-4b1d-b443-63e57e2aac55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f965c9163ebf41a9866af7389feaf17c)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:50:17.985896 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c5e73-9251-4b1d-b443-63e57e2aac55 admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:50:18.031872 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-d14c5e73-9251-4b1d-b443-63e57e2aac55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f965c9163ebf41a9866af7389feaf17c'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:50:18.038817 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-d14c5e73-9251-4b1d-b443-63e57e2aac55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f965c9163ebf41a9866af7389feaf17c'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:50:18.046222 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-d14c5e73-9251-4b1d-b443-63e57e2aac55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f965c9163ebf41a9866af7389feaf17c'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:50:18.058781 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-d14c5e73-9251-4b1d-b443-63e57e2aac55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f965c9163ebf41a9866af7389feaf17c'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:50:18.059446 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-d14c5e73-9251-4b1d-b443-63e57e2aac55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f965c9163ebf41a9866af7389feaf17c'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:50:18.065253 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1445/2900] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:50:17 2022] DELETE /identity/v3/users/f965c9163ebf41a9866af7389feaf17c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:50:18.244514 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9339fa-ee84-44a9-9bf5-c210dc9458d4 service neutron] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:50:18.254693 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9339fa-ee84-44a9-9bf5-c210dc9458d4 service neutron] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:50:18.311460 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e5cc9-b561-4f16-a7cb-628426f0897c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:50:18.312026 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e5cc9-b561-4f16-a7cb-628426f0897c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:50:18.312302 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e5cc9-b561-4f16-a7cb-628426f0897c admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:50:18.312575 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e5cc9-b561-4f16-a7cb-628426f0897c admin admin] PATH_INFO: `/v3/projects/c24a78af0a5d46de89b42a548fc3f43c` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:50:18.313747 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e5cc9-b561-4f16-a7cb-628426f0897c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c24a78af0a5d46de89b42a548fc3f43c)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:50:18.315630 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e5cc9-b561-4f16-a7cb-628426f0897c admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:50:18.361886 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-638e5cc9-b561-4f16-a7cb-628426f0897c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c24a78af0a5d46de89b42a548fc3f43c', 'request_id': 'req-638e5cc9-b561-4f16-a7cb-628426f0897c'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:50:18.369034 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-638e5cc9-b561-4f16-a7cb-628426f0897c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c24a78af0a5d46de89b42a548fc3f43c', 'request_id': 'req-638e5cc9-b561-4f16-a7cb-628426f0897c'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:50:18.374462 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-638e5cc9-b561-4f16-a7cb-628426f0897c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c24a78af0a5d46de89b42a548fc3f43c', 'request_id': 'req-638e5cc9-b561-4f16-a7cb-628426f0897c'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:50:18.385624 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-638e5cc9-b561-4f16-a7cb-628426f0897c admin admin] The token cache is being invalidate because project c24a78af0a5d46de89b42a548fc3f43c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:50:18.385908 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-638e5cc9-b561-4f16-a7cb-628426f0897c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:50:18.386519 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-638e5cc9-b561-4f16-a7cb-628426f0897c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:50:18.388016 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1456/2901] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:50:18 2022] DELETE /identity/v3/projects/c24a78af0a5d46de89b42a548fc3f43c => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:50:18.392055 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c5e73-9251-4b1d-b443-63e57e2aac55 admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:50:18.393468 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d14c5e73-9251-4b1d-b443-63e57e2aac55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:50:18.407123 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c5e73-9251-4b1d-b443-63e57e2aac55 admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:50:18.460346 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ba86f-65bc-4771-83b4-bf3c3259332f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:50:18.461111 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ba86f-65bc-4771-83b4-bf3c3259332f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:50:18.462343 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ba86f-65bc-4771-83b4-bf3c3259332f admin admin] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:50:18.462624 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ba86f-65bc-4771-83b4-bf3c3259332f admin admin] PATH_INFO: `/v3/users/1b17a8867702402eafa70062443c46c9` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:50:18.463779 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ba86f-65bc-4771-83b4-bf3c3259332f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b17a8867702402eafa70062443c46c9)` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:50:18.465708 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ba86f-65bc-4771-83b4-bf3c3259332f admin admin] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:50:18.510633 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-902ba86f-65bc-4771-83b4-bf3c3259332f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b17a8867702402eafa70062443c46c9'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:50:18.517524 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-902ba86f-65bc-4771-83b4-bf3c3259332f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b17a8867702402eafa70062443c46c9'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:50:18.524267 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-902ba86f-65bc-4771-83b4-bf3c3259332f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b17a8867702402eafa70062443c46c9'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:50:18.531832 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:50:18.536701 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-902ba86f-65bc-4771-83b4-bf3c3259332f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b17a8867702402eafa70062443c46c9'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:50:18.537521 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.notifications [None req-902ba86f-65bc-4771-83b4-bf3c3259332f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b17a8867702402eafa70062443c46c9'} {{(pid=80110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:50:18.544191 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1446/2902] 10.222.0.74 () {58 vars in 1224 bytes} [Wed Jul 27 16:50:18 2022] DELETE /identity/v3/users/1b17a8867702402eafa70062443c46c9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 16:50:18.711894 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e5cc9-b561-4f16-a7cb-628426f0897c admin admin] Authenticating user token {{(pid=80109) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:50:18.713370 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.fernet_utils [None req-638e5cc9-b561-4f16-a7cb-628426f0897c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80109) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:50:18.728585 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e5cc9-b561-4f16-a7cb-628426f0897c admin admin] Validating token access rules against request {{(pid=80109) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:50:18.751720 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:50:18.790889 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d494f-2621-4891-9daf-4a1b1fc9d962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2682f5470265422cbaaa91dfd16e8ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d94f0c6f55c441591df5c2cef40a859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80109) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:50:18.791538 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d494f-2621-4891-9daf-4a1b1fc9d962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:50:18.791821 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d494f-2621-4891-9daf-4a1b1fc9d962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:50:18.792101 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d494f-2621-4891-9daf-4a1b1fc9d962 admin admin] PATH_INFO: `/v3/projects/68d55b4deef34beba72a1f5971bed410` {{(pid=80109) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:50:18.793291 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d494f-2621-4891-9daf-4a1b1fc9d962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68d55b4deef34beba72a1f5971bed410)` {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:50:18.795199 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d494f-2621-4891-9daf-4a1b1fc9d962 admin admin] RBAC: Authorization granted {{(pid=80109) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:50:18.840288 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-461d494f-2621-4891-9daf-4a1b1fc9d962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68d55b4deef34beba72a1f5971bed410', 'request_id': 'req-461d494f-2621-4891-9daf-4a1b1fc9d962'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:50:18.847037 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-461d494f-2621-4891-9daf-4a1b1fc9d962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68d55b4deef34beba72a1f5971bed410', 'request_id': 'req-461d494f-2621-4891-9daf-4a1b1fc9d962'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:50:18.854229 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-461d494f-2621-4891-9daf-4a1b1fc9d962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68d55b4deef34beba72a1f5971bed410', 'request_id': 'req-461d494f-2621-4891-9daf-4a1b1fc9d962'} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:50:18.870179 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-461d494f-2621-4891-9daf-4a1b1fc9d962 admin admin] The token cache is being invalidate because project 68d55b4deef34beba72a1f5971bed410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80109) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 16:50:18.870553 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-461d494f-2621-4891-9daf-4a1b1fc9d962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:50:18.871398 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG keystone.notifications [None req-461d494f-2621-4891-9daf-4a1b1fc9d962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80109) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 16:50:18.873754 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: [pid: 80109|app: 0|req: 1457/2903] 10.222.0.74 () {58 vars in 1233 bytes} [Wed Jul 27 16:50:18 2022] DELETE /identity/v3/projects/68d55b4deef34beba72a1f5971bed410 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 16:50:28.541156 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:50:28.865558 dv-cinder-849690-6-iscsi devstack@keystone.service[80109]: DEBUG dbcounter [-] [80109] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80109) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 16:54:59.039825 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ba86f-65bc-4771-83b4-bf3c3259332f admin admin] Authenticating user token {{(pid=80110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 16:54:59.041279 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-902ba86f-65bc-4771-83b4-bf3c3259332f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:54:59.180040 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ba86f-65bc-4771-83b4-bf3c3259332f admin admin] Validating token access rules against request {{(pid=80110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 16:54:59.237745 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfd837-389a-49e5-be41-20ab273451bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91f8e0f82a8541aead6cd8cc45f4a86b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d6bcdd09cf24a5c8003951db37020cf', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 16:54:59.238366 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfd837-389a-49e5-be41-20ab273451bd service placement] REQUEST_METHOD: `GET` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 16:54:59.238646 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfd837-389a-49e5-be41-20ab273451bd service placement] SCRIPT_NAME: `/identity` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 16:54:59.238918 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfd837-389a-49e5-be41-20ab273451bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 16:54:59.240742 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.fernet_utils [None req-d5cfd837-389a-49e5-be41-20ab273451bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 16:54:59.252642 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfd837-389a-49e5-be41-20ab273451bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 16:54:59.254249 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfd837-389a-49e5-be41-20ab273451bd service placement] RBAC: Authorization granted {{(pid=80110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 16:54:59.264017 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: [pid: 80110|app: 0|req: 1447/2904] 10.222.0.74 () {60 vars in 1365 bytes} [Wed Jul 27 16:54:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 16:55:09.259878 dv-cinder-849690-6-iscsi devstack@keystone.service[80110]: DEBUG dbcounter [-] [80110] Writing DB stats keystone:SELECT=89 {{(pid=80110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}